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Long-term  research  objective: 

The  verification  and  validation  (V&V)  of  simulations  is  a  subject  of  great  current 
interest  across  the  entire  spectrum  of  computational  mechanics.  V&V  techniques 
are  in  their  infancy  and  are  the  subjects  of  spirited  debate.  Our  objective  was  to 
develop  a  rigorous  V&V  methodology  that  (1)  properly  includes  the  effects  of 
uncertainties  in  the  simulations  and  in  the  experimental  validation  data  and  (2) 
provides  a  quantitative  assessment  of  the  degree  of  (or  lack  of)  validation  of  a 
simulation  for  a  specific  configuration  and  condition. 

S&T  objective: 

Develop  V&V  techniques  for  Computational  Fluid  Dynamics  (CFD)  codes  and 
Structural  Loads  and  Motions  codes  supported/developed  by  ONR.  Participate 
in  the  application  of  the  V&V  techniques  to  assess  the  codes  and  produce 
documented  solutions  for  specific  configurations  and  specific  conditions. 

Results: 

The  methodology  has  been  refined  and  applied  in  the  ONR  programs 
“VerificationA/alidation  of  Computational  Ship  Hydrodynamics,”  “International 
Collaboration  on  Complementary  Computational  and  Experimental  Ship 
Hydrodynamics  and  Uncertainty  Analysis,”  and  “Surface  Combatant  Accelerated 
Hydrodynamics  S&T  Initiative”.  These  programs  include  collaborative  efforts 
among  multiple  universities,  multiple  industrial  consortia,  U.S.  government 
researchers,  and  researchers  at  the  Italian  Istituto  Nazionale  Per  Studi  Ed 
Esperienze  Di  Architettura  Navale.  The  results  are  documented  in  the  following 
Appendices: 

Appendix  A 

Coleman,  H.  W.  and  Stern,  F. ,  "Uncertainties  and  CFD  Code  Validation,"  J. 
Fluids  Engineering,  Vol.  119,  p.  795-803,  December  1997.  (Also 
“Discussion  and  Authors’  Closure,”  Vol.120,  p  635-6,  September  1998.) 

Appendix  B 

Stern,  F.,  Longo,  J.,  Penna,  R.,  Olivieri,  A.,  Ratcliffe,  T.,  and  Coleman,  H., 
“International  Collaboration  on  Benchmark  CFD  Validation  Data  for 
Surface  Combatant  DTMB  Model  5415,”  Proceedings  of  the  23^“ 
Symposium  on  Naval  Hydrodynamics,  Val  de  Reuil,  France,  September 
17-22,  2000. 
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Appendix  C 

Coleman,  H.  W.,  Stern,  F.,  Di  Mascio,  A.,  and  Campana,  E.  "The  Problem 
With  Oscillatory  Behavior  in  Grid  Convergence  Studies,”  J.  Fluids 
Engineering,  Vol.  123,  No.  2,  p438-439,  June  2001. 

Appendix  D 

Stern,  R,  Wilson,  R.  V.,  Coleman,  H.  W.,  and  Paterson,E.  G., 
“Comprehensive  Approach  to  Verification  and  Validation  of  CFD 
Simulations  -  Part  1;  Methodology  and  Procedures,”  J.  Fluids 
Engineering,  Vol.  123,  No.  4,  December  2001. 

Appendix  E 

Wilson,  R.  V.,  Stern,  R,  Coleman,  H.  W.,  and  Paterson, E.  G., 
“Comprehensive  Approach  to  Verification  and  Validation  of  CFD 
Simulations  -  Part  2;  Application  for  RANS  Simulation  of  a 
Cargo/Container  Ship,”  J.  Fluids  Engineering,  Vol.  123,  No.  4,  December 
2001. 
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Uncertainties  and  CFD 
Code  Validation 

A  new  approach  to  computational  fluid  dynamics  code  validation  is  developed  that 
gives  proper  consideration  to  experimental  and  simulation  uncertainties.  The  compar¬ 
ison  error  is  defined  as  the  difference  between  the  data  and  simulation  values  and 
represents  the  combination  of  all  errors.  The  validation  uncertainty  is  defined  as  the 
combination  of  the  uncertainties  in  the  experimental  data  and  the  portion  of  the 
uncertainties  in  the  CFD  prediction  that  can  be  estimated.  This  validation  uncertainty 
sets  the  level  at  which  validation  can  be  achieved.  The  criterion  for  validation  is  th^ 
the  magnitude  of  the  comparison  error  must  be  less  than  the  vdidation  uncertainty. 
If  validation  is  not  accomplished,  the  magnitude  and  sign  of  the  comparison  error  can 
be  used  to  improve  the  mathematical  modeling.  Consideration  is  given  to  validation 
procedures  for  a  single  code,  multiple  codes  and/or  models,  and  predictions  of  trends. 
Example  results  of  verification/validation  are  presented  for  a  single  computational 
fluid  dynamics  code  and  for  a  comparison  of  multiple  turbulence  models.  The  results 
demonstrate  the  usffulness  of  the  proposed  validation  strategy.  This  new  approach 
for  validation  should  be  useful  in  guiding  future  developments  in  computational  fluid 
dynamics  through  validation  studies  and  in  the  transition  of  computational  fluid 
dynamics  codes  to  design. 


1  Introduction 

Uncertainty  considerations  involved  in  using  experimental 
data  in  validating  the  predictions  of  CFD  codes  (or,  more  gener¬ 
ally,  computer  simulations)  are  discussed  in  this  article.  The 
word  uncertainty  is  used  in  the  following  sense — the  uncer¬ 
tainty  U  associated  with  a  measured  quantity  or  a  predicted 
quantity  defines  the  ±U  interval  about  that  quantity  within 
which  we  expect  the  true  (but  unknown)  value  of  that  quantity 
to  lie  95  times  out  of  100.  It  is  important  to  recognize  that  a 
validation  is  restricted  to  some  range,  typically  the  range  of 
conditions  of  the  data  used  in  the  (success&l)  validation  effort 
This  is  intended  by  the  authors  to  be  implicit  in  the  discussions 
in  this  article. 

The  comparison  process  in  attempting  to  validate  predictions 
using  experimental  data  is  illustrated  schematically  in  Fig.  1. 
The  hypothetical  predictions  of  result  r  versus  independent  vari¬ 
able  X  from  two  models  (or  simulations)  are  shown  along  with 
experimental  data  points  (X^ ,  r, ).  In  part  (a)  of  the  figure,  no 
uncertainties  are  considered,  and  one  might  well  be  tempted  to 
argue  that  Model  1  is  superior  to  Model  2.  The  predictions  from 
Model  1  seem  to  ‘‘capture  the  trend  of  the  data’*  better  than 
the  predictions  from  the  (simplistic,  linearized,  etc.)  Model  2. 
If  the  uncertainties  in  the  experimentally-determined  values  of 
the  result  r  are  considered  (Hg.  1  (b)),  the  perspective  changes 
completely,  and  it  is  obvious  that  arguing  for  one  method  over 
another  based  on  comparison  with  the  experimental  data  is 
wasted  effort  since  the  predictions  from  both  methods  fall  well 
within  the  data  uncertainty. 

Actually,  Fig.  l{b)  does  not  show  the  complete  situation. 
For  the  data  points,  uncertainties  in  both  the  experimentally- 
determined  r  and  the  experimentally-determined  value  of  the 
independent  variable  X  should  be  considered,  giving  an  uncer¬ 
tainty  “box”  around  each  experimental  data  point  Addition¬ 
ally,  the  prediction  from  a  m^el  should  not  be  viewed  as  an 
infinitesimally  thin  r  vs.  X  line,  but  rather  as  a  “fuzzy  band” 
that  represents  the  prediction  plus  and  minus  the  uncertainty 
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that  should  be  associated  with  the  simulation/model /code.  This 
is  illustrated  in  Fig.  2,  which  also  shows  that,  in  general,  uncer¬ 
tainties  in  both  the  data  and  the  predictions  can  vary  ( sometimes 
dramatically)  over  the  range  of  X.  Hgure  2  shows  the  variables 
and  their  uncertainties,  but  the  comparison  it  shows  is  deceptive 
because  it  is  two-dimensional.  The  independent  variable  X  must 
be  considered  a  vector  (X )  of  n  dimensions — fluid  velocity  as 
a  function  of  position  and  time,  ^^(jc,  y,  z,  t),  for  example — 
and  the  “box”  around  X  will  therefore  be  n-imensional.  The 
(total)  uncertainty  in  r  that  should  be  used  in  a  conqiarison 
should  include  the  experimental  uncertainty  in  r  and  the  addi¬ 
tional  uncertainty  in  r  arising  firom  experimental  uncertainties 
in  the  measurements  of  the  n  independent  variables  (this  is 
developed  in  detail  in  Section  3). 

Contributors  to  the  prediction  uncertainty  Ur(X)  can  be  di¬ 
vided  into  two  broad  categories — numerical  uncertainty  and 
modeling  uncertainty.  In  fact,  insuring  that  the  modeling  uncer¬ 
tainty  is  below  some  designated  value  is  one  purpose  of  CFD 
validation  through  comparisons  with  benchmark  experimental 
data. 

The  validation  strategy  proposed  in  this  article  and  discussed 
in  detail  in  Section  3  views  the  situation  from  a  new  perspective, 
isolating  the  modeling  uncertainty  (which  the  authors  do  not 
know  how  to  estimate)  from  the  uncertainties  that  can  be  esti¬ 
mated  (the  data  uncertainty  and  the  non-modeling  uncertainties 
in  predictions).  A  direct  calculation  of  the  comparison  error  E 
(data  minus  prediction)  is  made  and  compared  with  a  validation 
uncertainty  Uv  that  is  composed  of  the  uncertainties  in  the  ex¬ 
perimental  data  and  the  portion  of  the  uncertainties  in  the  CFD 
prediction  that  can  be  estimated.  This  validation  uncertainty  Uv 
is  the  best  resolution  possible  in  the  validation  e:^ort  (i.e.,  it  sets 
the  “noise  level”  below  which  no  discrimination  is  possible).  If 
the  absolute  value  of  the  calculated  comparison  error  E  is  less 
than  Uv,  then  validation  is  defined  as  being  successful  at  the 
Uv  level. 

From  the  preceding  discussion  the  authors  believe  it  is  evi¬ 
dent  that  (1)  the  uncertainties  in  the  data  and  in  the  predictions 
set  the  scale  at  which  validation  is  possible,  and  (2)  these 
uncertainties  must  be  considered  in  determining  if  validation 
has  been  achieved.  Obviously,  these  uncertainties  should  be 
considered  in  planning  and  implementing  a  computational/ex¬ 
perimental  research  program  for  validating  CTO  codes,  al- 
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Fig.  1  {b)  with  partial  consideration  of  experimental  uncertainties 
Fig.  1  Comparisons  of  experimental  data  and  model  predictions 


Fig.  2  Uncertainties  in  data  and  predictions  to  be  considered  in  valida¬ 
tions 


though  they  typically  have  not  been  in  the  past.  Figure  1(a) 
gives  a  qualitative  view  of  the  way  most  previous  validation 
efforts  have  proceeded,  with  Fig.  1  (^)  being  typical  of  the  few 
approaches  considering  uncertainties  at  all. 

This  cuirent  work  is  part  of  a  larger  program  (Rood,  1996) 
for  developing  and  implementing  a  strategy  for  validation  of 
Reynolds- Averaged  Navier-Stokes  (RANS )  computational  ship 
hydrodynamics  codes.  The  program  includes  complementary 
computational  ship  hydrodynamics  and  towing  tank  investiga¬ 


tions  and  considers  the  uncertainties  in  both  computations  and 
data  in  assessing  the  success  of  validation  efforts. 

2  Uncertainties  in  Data  and  in  Simulations 

As  stated  earlier,  the  uncertainty  U  associated  with  a  mea¬ 
sured  quantity  or  a  predicted  quantity  defines  tht  ±U  interval 
about  Aat  quantity  within  which  we  expect  the  true  (but  un¬ 
known)  value  of  that  quantity  to  lie  95  times  out  of  100.  For 
detailed  discussion  of  uncertainties  associated  with  experimen¬ 
tal  data,  the  reader  is  referred  to  Coleman  and  Steele  (1989, 
1995).  Although  the  uncertainty  in  an  experimental  data  point 
originally  comes  from  both  bias  (systematic)  and  precision 
(random)  sources,  the  uncertainty  is  “fossilized”  into  a  fixed 
quantity  (a  bias)  once  the  value  (X/,  r/)  of  the  data  point  is 
recorded  and  reported.  This  is  logical  if  one  notes  that  the  value 
of  the  data  point  is  always  the  same  amount  different  from  the 
(unknown)  true  value  each  time  the  data  point  is  used. 

The  uncertainties  associated  with  the  predictions  of  models, 
simulations,  CFD  codes,  etc.  and  their  role  in  the  validation 
process  have  been  receiving  increasing  attention  in  the  last  few 
years,  and  only  a  few  representative  references  are  cited  here. 
Editorial  policies  have  been  set  by  professional  journals 
(ASME,  1993;  AIAA,  1994;  Gresho  and  Taylor,  1994),  and 
standards  and  guidelines  (lAHR,  1994)  and  recommended  prac¬ 
tices  (me,  1996)  have  been  specified  by  international  organi¬ 
zations.  The  literature  covers  a  broad  range  from  governmental 
(Rood,  1996)  and  industrial  (Melnik  et  al.,  1994)  perspectives 
to  overall  methodology  (Coleman,  1996;  Marvin,  1995;  Mehta, 
1996  (also  AIAA,  1997);  Oberkampf  et  al.,  1995)  and  detailed 
application  (Blotmer,  1990;  Roache,  1997;  Zingg,  1992).  The 
1993  ASME  Symposium  on  Quantification  of  Uncertainty  in 
Computational  Fluid  Dynamics  (Celik  et  al.,  1993)  provides  a 
good  introduction. 

As  mentioned  in  the  previous  section,  uncertainties  associated 
with  predictions  from  simulations  can  be  divided  into  two  broad 
categories:  (1)  numerical  uncertainties,  and  (2)  modeling 
uncertainties.  The  numerical  uncertainty  category  includes  un¬ 
certainties  due  to  the  numerical  solution  of  the  mathematical 
equations  (discretization,  artificial  dissipation,  iterative  and  grid 
non-convergence,  local  and  global  non-conservation  of  mass, 
moinentum,  energy,  etc.,  internal  and  external  boundary  non¬ 
continuity,  computer  round-off,  etc.)  The  modeling  uncertainty 
category  includes  uncertainties  due  to  assumptions  and  approxi¬ 
mations  in  the  mathematical  representation  of  the  physical  pro¬ 
cess  (geometry,  mathematical  equation,  coordinate  transforma¬ 
tion  approximations,  free-surface  boundary  conditions,  turbu¬ 
lence  models,  etc)  and  also  uncertainties  due  to  the 
incorporation  of  previous  experimental  data  into  the  model 
(such  things  as  fluid  property  values  and  the  “constants”  in 
turbulence  models).  Examples  of  reported  uncertainties  associ¬ 
ated  with  property  data  range  from  0.25-0.5  percent  for  liquid 
oxygen  density  (Brown  et  al.,  1994),  to  2-5  percent  for  the 
thermal  conductivity  of  air  at  atmospheric  pressure  (Coleman 
and  Steele,  1989),  to  huge  percentages  for  properties  such  as 
surface  tension  coefficient  that  are  extremely  sensitive  to  con¬ 
taminants.  A  recent  study  (Beard  and  Landrum,  1996)  utilizing 
laminar  Navier-Stokes  computations  for  hydrogen  flow  through 
a  solar  thermal  thruster  at  temperatures  up  to  6100  R  showed 
a  i2  percent  variation  in  computed  specific  impulse  due  solely 
to  the  range  of  the  available  reaction  rate  data  reported  by 
different  investigators. 

The  overall  process  leading  to  validation  and  simulation  un- 
certmnty  estimation  can  be  categorized  as  documentation,  veri¬ 
fication,  and  validation.  Documentation  involves  detailed  pre¬ 
sentation  of  the  mathematical  equations  and  numerical  method. 
Verification  involves  estimation  of  numerical  uncertainty 
through  parametric,  convergence,  and  order-of-accuracy  stud¬ 
ies.  Validation  involves  estimation  of  the  difference  (error) 
between  the  simulation’s  prediction  and  the  truth,  and  this  esti- 
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Fig.  3  Definition  of  comparison  error 


mate  is  impossible  to  make  with  any  coniidence  without  a 
benchmark.  The  benchmark  can  be  an  analytical  solution  (with 
an  associated  uncertainty)  or,  more  likely  and  of  primary  inter¬ 
est  here,  an  experimental  value  with  its  associated  uncertainty. 
To  parsqjhrase  one  reviewer,  (verification)/(validation)  can  be 
viewed  as  addressing  (equations  solved  ri^t?)/(right  equations 
solved?). 

Although  not  always  available,  documentation  is  relatively 
strai^tforward,  whereas,  in  spite  of  the  aforementioned  efforts, 
specific  implementation  procedures  for  verification  and  valida¬ 
tion  are  not  yet  established.  Approaches  for  verification  require 
procedures  for  the  estimation  of  the  numerical  uncertainties. 
Stem  et  al.  (1996)  provided  an  example  approach  for  their 
steady  RANS  CFD  method  with  application  to  naval  surface 
combatants.  In  this  approach,  estimates  of  uncertainties  were 
provided  for  both  integral  and  point  quantities  for  iterative  and 
grid  nonconvergence  and  were  combined  using  root  sum  square. 
Also,  for  conditions  permitting,  order-of-accuracy  and  Richard¬ 
son  extrapolation  studies  were  conducted. 

In  the  following  section,  a  new  approach  to  CFD  validation 
is  developed  and  discussed  with  regard  to  validation  of  a 
single  CFD  code,  to  validation  of  a  comparison  of  multiple 
codes  and/or  models,  and  to  validation  of  predictions  of 
trends.  Subsequently,  in  Section  4,  example  results  of  valida¬ 
tions  are  presented  both  for  a  single  CFD  code  and  for  a 
comparison  of  multiple  turbulence  models.  The  CFD  code 
and  verification  procedures  of  Stem  et  al.  (1996)  are  used 
for  two  applications  for  which  numerical  and  experimental 
uncertainty  analyses  are  available:  marine-propulsor  flow 
(Chen,  1996;  Jessup,  1994)  and  two-dimensional  turbulent 
fiat-plate  boundary-layer  flow  (Sreedhar  and  Stem,  1997; 
Longo,  et  al.,  1998). 

3  An  Approach  to  CFD  Code  Validation 

Consider  the  situation  shown  in  Fig.  3.  Using  the  example 
mentioned  previously,  the  single-plane  representation  r  versus 
X might  be  a  mean  velocity  component  Vvs.  distance  (z)  normal 
to  a  solid  surface  at  a  given  time  and  position  (x,  y,  t)  on  that 
surface.  Define  the  predicted  r-value  from  the  simulation  (code) 
as  S,  the  experimentally  determined  r-value  of  the  (X^ ,  r, )  data 
point  as  D,  and  the  comparison  error,  £,  as  their  difference: 

E^D-S  (1) 

The  comparison  error  E  is  the  resultant  of  all  of  the  errors 
associated  with  the  experimental  data  and  the  errors  associ¬ 
ated  with  the  simulation.  Here  it  is  assumed  that  a  correction 
has  been  made  for  any  error  whose  value  is  known.  Thus, 
the  errors  that  are  the  subject  of  this  discussion  have  unknown 
sign  and  magnitude,  and  the  uncertainties  are  estimates  of 
these  errors. 


If  X,  ,  r/,  and  5  share  no  common  error  sources,  then  the 
uncertainty  Ug  in  the  comparison  error  can  be  expressed  as 

where  Ud  is  the  uncertainty  in  the  data  and  Us  the  uncertainty 
in  the  simulation.  The  uncertainty  Ue  should  bound  the  (true) 
absolute  value  of  the  comparison  error  E  95  times  out  of  100. 
The  assumptions  and  approximations  made  in  deriving  Eq.  (2) 
are  discussed  in  detail  in  Coleman  and  Steele  (1995). 

Recalling  the  discussion  in  Section  2,  the  simulation  uncer¬ 
tainty  Us  can  be  represented  as 

Uho  +  UluA  (3) 

where  Uss  is  the  simulation  numerical  solution  uncertainty,  Uspd 
is  the  simulation  modeling  uncertainty  arising  from  using  previ¬ 
ous  experimental  data,  and  Usm  is  the  simulation  modeling 
uncertainty  arising  from  modeling  assumptions.  Substimting 
Eq.  (3)  into  Eq.  (2)  gives 

Ue  =  Up  +  UsN  +  Uspd  +  UIma  (4) 

Ideally,  we  would  like  to  postulate  that  if  the  absolute  value 
of  E  is  less  than  its  uncertainty  Ue,  then  validation  is  achieved. 
In  reality,  the  authors  know  of  no  approach  that  gives  an  esti¬ 
mate  of  UsMA  ♦  so  Ue  cannot  be  estimated.  That  leaves  a  more 
stringent  validation  test  as  the  practical  alternative.  If  we  define 
the  validation  uncertainty  Uv  as  the  combination  of  all  uncer¬ 
tainties  that  we  know  how  to  estimate  (i.e.,  all  but  Usma),  then 

^Ul-UlMA  =  Ui+Uh+  UjpD  (5) 

If  I  £|  is  less  than  the  validation  uncertainty  Uv,  then  the 
combination  of  aU  the  errors  in  D  and  5  is  smaller  than  the 
estimated  validation  uncertainty  and  validation  has  been 
achieved  at  the  Uy  level.  This  quantity  Uv  is  the  key  metric  in 
the  validation  process.  Uv  is  tiie  validation  “noise  level”  im¬ 
posed  by  the  uncertainties  inherent  in  the  data,  the  numerical 
solution,  and  the  previous  experimental  data  used  in  the  simula¬ 
tion  model — one  cannot  discriminate  once  |  £]  is  less  than  this, 
i.e.,  as  long  as  |  £|  is  less  than  this  one  cannot  evaluate  the 
effectiveness  of  proposed  model  “iiiq)rovements.”  Choice  of 
the  required  level  of  Uy  is  associated  with  the  degree  of  risk 
deemed  acceptable  in  a  program. 

To  estimate  Uspd  for  a  case  in  which  the  simulation  uses 
previous  data  di  in  m  instances,  one  would  need  to  evaluate 

where  the  Ud^  would  be  estimated  using  established  uncertainty 
analysis  procedures  (Coleman  and  Steele,  1989,  1995). 

As  discussed  in  Section  1,  for  the  data  point  (X^,  r,  ),  Ud 
should  include  both  the  experimental  uncertainty  in  r/  and  the 
additional  uncertainties  in  r,-  arising  from  experimental  uncer¬ 
tainties  in  the  measurements  of  the  n  independent  variables  (Xj)i 
in  X, .  The  expression  for  Ud  that  should  be  used  in  the  Uy 
calculation  is  then 

In  some  cases,  the  terms  in  the  summation  in  Eq.  (7)  may  be 
shown  to  be  very  small  using  an  order-of-magnitude  analysis 
and  then  neglected.  This  would  occur  in  situations  in  which 
the  Ux/s  are  of  “reasonable”  magnitude  and  gradients  in  r 
are  sniall.  In  regions  of  the  flow  with  high  gradients  (near  a 
surface  in  a  turbulent  flow),  however,  these  terms  may  be  very 
significant. 
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There  is  also  a  very  real  possibility  that  measurements  of 
different  variables  might  share  identical  bias  errors.  This  is  easy 
to  imagine  for  measurements  of  jc,  y,  and  z*  Another  possibility 
is  D  and  S  sharing  an  identical  error  source,  for  example  if  r  is 
drag  coefficient  and  the  same  density  table  (curvefit)  is  used 
both  in  data  reduction  in  the  experiment  and  in  the  simulation. 
In  such  cases,  additional  “correlated  bias”  terms  must  be  in¬ 
cluded  in  Eq.  (2).  Approximation  and  inclusion  of  such  terms 
are  discussed  in  Coleman  and  Steele  (1989,  1995),  Coleman 
et  al.  ( 1995),  and  Brown  et  al.  ( 1996)  and  will  not  be  covered 
in  fiirdier  detail  in  this  article. 

Validation  of  a  Single  CFD  Code.  The  validation  uncer¬ 
tainty,  Uv,  sets  the  level  at  which  validation  can  be  achieved. 
If  the  objectives  of  a  program  require  that  validation  be  accom¬ 
plished  “within  ±2  percent,”  then  |£|  must  be  less  than  Uv 
and  Uv  must  be  less  than  (roughly)  0.02  D.  If  Uv  is  greater 
than  0.02  D,  the  objectives  of  the  program  cannot  be  achieved 
until  the  sum  of  the  terms  on  the  right-hand-side  of  Eq.  (5)  is 
reduced  to  an  acceptable  level.  When  |  E\  is  greater  than  C/v, 
validation  is  not  accomplished,  and  the  magnitude  and  sign 
of  E  can  be  valuable  in  designing  strategies  to  improve  the 
mathematical  modeling. 

As  one  reviewer  pointed  out,  consideration  of  Eq.  (5)  shows 
that  (a)  the  more  uncertain  the  data,  and/or  (b )  the  more  inaccu¬ 
rate  the  code  (greater  Usn  and  Uspd)*  the  easier  it  is  to  validate 
a  code.  That  is  true,  since  the  greater  the  uncertainties  in  the 
data  and  the  code  predictions,  the  greater  the  “noise  level” 
{Uv)-  If  this  value  of  Uv  is  greater  than  that  designated  as 
necessary  in  a  research/ design/ development  program,  however, 
then  the  required  level  of  validation  could  not  be  achieved  with¬ 
out  improvement  in  the  quality  of  the  data,  the  code,  or  both. 
Likewise,  if  Uss  and  Uspo  are  not  estimated  but  |  E\  is  less  than 
Uoy  then  validation  has  been  achieved  but  at  an  urdenown  level 
Obviously,  if  no  uncertainties  are  estimated,  no  statements  about 
validation  can  be  made  within  the  concept  of  the  validation 
process  as  considered  in  this  article. 

In  general,  validation  of  a  code’s  predictions  of  a  number 
(N)  of  different  variables  is  desired,  and  this  means  that  in 
a  particular  validation  effort  there  will  be  N  different  Fs  and 
f/v’s  and  (perhaps)  some  successful  validations  and  some 
unsuccessful.  For  each  variable,  a  plot  of  the  simulation  pre¬ 
diction  versus  X  compared  with  the  (X/ ,  r/ )  data  points  gives 
a  traditional  overview  of  the  validation  status,  but  the  inter¬ 
pretation  of  the  comparison  is  greatly  affected  by  choice  of 
the  scale  and  the  size  of  the  symbols.  A  plot  of  ±Uv  and  E 
versus  X  for  each  variable  is  particularly  useful  in  drawing 
conclusions,  as  demonstrated  in  Section  4,  and  the  interpreta¬ 
tion  of  the  comparison  is  more  insensitive  to  scale  and  symbol 
size  choices. 

Comparison  of  Multiple  Codes  and/or  Models  in  a  Vali¬ 
dation  Effort  When  a  validation  effort  involves  multiple 
codes  and/or  models,  the  procedure  discussed  above — compar¬ 
ison  of  the  Fs  and  £/v’s  for  the  N  variables — should  be  per¬ 
formed  for  each  code/model. 

Since  each  code/model  may  have  a  different  Fv,  some 
method  to  compare  the  different  codes’ /models’  performance 
for  each  variable  in  the  validation  is  useful.  The  range  within 
which  (95  times  out  of  100)  the  true  value  of  E  lies  is  £  ±  Ue, 
From  1^.  (5),  when  Usma  is  zero  then  Uv  =  Ue,  so  for  that 
ideal  condition  the  maximum  absolute  magnitude  of  the  95% 
confidence  interval  is  given  by  |  £|  +  Uv-  Comparison  of  the 
(I  £|  +  Uvfs  for  the  different  c^es/models  then  shows  which 
has  the  smallest  range  of  likely  error  assuming  all  Usma's  are 
zero.  This  allows  appropriate  comparisons  of  (low  £)/(high 
Uv)  with  (high  £/iow  Uv)  codes/models. 

Validation  of  Predictions  of  Trends*  In  some  instances, 
the  ability  of  a  code  or  model  to  predict  the  trend  of  a  variable 
may  be  the  subject  of  a  validation  effort.  An  exanqple  would 


be  the  difference  in  drag  for  two  ship  configurations  tested  at 
the  same  Froude  number.  The  procedure  discussed  above — 
comparison  of  |  £  |  and  Uv  for  the  drag — should  be  performed 
for  each  configuration.  The  difference  6  in  drag  for  the  two 
configurations  should  then  be  considered  as  the  variable  that  is 
the  subject  of  the  validation.  As  discussed  in  detail  in  Coleman 
et  al.  (1995),  because  of  correlated  bias  uncertainty  effects  in 
the  experimental  data  the  magnitude  of  the  uncertainty  in  6  may 
be  si^iificantly  less  than  the  uncertainty  in  either  of  the  two 
expeiimentaDy-determined  drag  values.  This  means  that  the 
value  of  Uv  for  6  may  be  significantly  less  than  the  Uv*s  for 
the  drag  values,  allowing  for  a  more  stringent  validation  crite¬ 
rion  for  the  difference  than  for  the  absolute  magnitudes  of  the 
variables. 

It  is  probable  that  in  some  instances  the  Usma  for  the  differ¬ 
ence  6  will  be  less  than  the  Usma*s  for  the  absolute  magnitudes 
of  the  variables  because  of  correlated  systematic  uncertainty 
effects  in  the  modeling.  This  is  an  unexplored  area  at  this  time. 

4  Results  of  Verification/Validlation  for  a  Single 
CFD  Code  and  for  a  Comparison  of  Multiple  Turbu¬ 
lence  Models 

Example  results  of  verification/validation  are  presented  both 
for  a  single  CFD  code  and  for  a  comparison  of  multiple  turbu¬ 
lence  models.  In  both  cases,  the  magnitude  of  Uspd  was  assumed 
negligible  relative  to  the  other  uncertainties.  The  CFD  code  and 
verification  procedures  of  Stem  et  al.  ( 1996)  are  used  for  two 
applications  for  which  numerical  and  experimental  uncertainty 
analyses  are  available:  marine-propulsor  flow  (Chen,  1996; 
Jessup,  1994)  and  two-dimensional  turbulent  fiat-plate  bound¬ 
ary-layer  flow  (Sreedhar  and  Stem,  1997;  Longo  et  al.,  1998). 
The  former  represents  a  practical  geometry  and  is  used  for 
the  single  CFD  code  validation  example,  whereas  the  latter 
represents  an  idealized  geometry  and  is  used  for  the  comparison 
of  turbulence  models  example. 

For  the  marine-propulsor  geometry,  Jessup  (1989)  provided 
an  extensive  data  set  (including  circv^erential-average,  phase- 
average,  and  detailed  blade  boundary-layer,  wake,  and  tip-vor¬ 
tex  velocities)  for  the  relatively  simple  marine  propulsor  P41 19. 
This  propeller-shaft  configuration,  shown  viewed  from  up¬ 
stream  in  Fig.  4,  was  tested  in  a  24  in.  water  tunnel  with  mea¬ 
surements  made  using  a  three-component  laser-Doppler  veloci- 
meter  (LDV)  system.  Detailed  uncertainty  estimates  have  also 
been  reported  (Jessup,  1994).  An  initial  validation  effort  was 


Fig.  4  Marine  propulsor  P4119 
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Table  1  Marine^propulsor  flow;  performance  coefficients 
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E% 
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Ud% 

Ur% 

Us^% 

Us,% 

UsG% 
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Kt 
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-2.1 

3.2 

2.0 

2.0 

2.5 

1.0 

2.3 

13 

IOKq 

0.280 

0.276 

1.9 

3.4 

2.0 

2.0 

2.7 

1.0 

2.5 

1.4 

%D. 


reported  by  Stem  et  al.  ( 1994) .  Documentation  and  verification 
are  reported  by  Chen  (1996)  in  conjunction  with  studies  of 
design  and  offi^esign  maiine-piopulsor  performance. 

For  the  two-dimensional  turbulent  flat-plate  boundary- 
layer  flow,  Longo  et  al.  (1996)  provided  data  and  uncertainty 
analysis  in  conjunction  with  their  study  of  solid/free-surface 
juncture  boundary  layer  and  wake.  A  1.2  m  surface-piercing 
flat  plate  was  tested  in  a  100  x  3  X  3  m  towing  tank  with 
measurements  made  using  a  two-component  LDV  system 
configured  to  obtain  three  mean  velocities  and  five  Reynolds 
stresses  for  both  boundary  layer  and  widce  planes  and  regions 
deep  and  very  close  to  the  toe  surface.  Sreedhar  and  Stem 
(1997)  provided  verification  for  multiple  turbulence  models 
for  this  application  in  conjunction  with  the  development  of 
nonlinear  eddy-viscosity  turbulence  models,  including  both 
wall  and  toe-surface  etocts. 

The  CFD  method  solves  the  unsteady  incompressible  RANS 
and  continuity  equations  using  either  noninertiai  cylindrical  or 
inertial  cartesian  coordinates  and  the  Baldwin-Lomax  turbu¬ 
lence  model.  The  RANS  equations  are  solved  using  finite-ana¬ 
lytic  spatial  and  first-order  (steady  flow)  or  second-order  (un¬ 
steady  flow)  backward  difference  time  discretization.  The  pres¬ 
sure  equation  is  derived  from  a  discretized  form  of  the 
continuity  equation  and  solved  using  second-order-central  finite 
differences.  The  overall  solution  procedure  is  based  on  the  two- 
step  pressure-implicit-split-operator  (PISO)  algorithm.  For 
steady  flow,  subiteration  convergence  is  not  required  and  time 
serves  as  an  iteration  parameter. 

Verification  Procedures.  The  verification  procedures  fol¬ 
low  the  approach  of  Stem  et  al.  ( 1996).  This  approach  is  based 
on  the  editorial  policy  statement  of  the  ASME  (ASME,  1993). 
The  ten  issues  of  the  statement  are  divided  as  documentation  ( 1, 
7, 8),  verification  (2-6),  and  validation  (9, 10).  Verification  is 
comprised  of  grid-,  iterative-,  and  time-convergence  (4,  5,  6), 
artificial  dissipation  (3),  and  order-of-accuracy  (2)  studies. 
These  studies  are  implemented  using  a  five-step  procedure:  (i) 
grid  design  and  identification  of  important  parameters;  (ii)  con¬ 
vergence  studies;  (iii)  determination  of  the  effects  of  explicit 
artificial  dissipation,  if  used;  (iv)  estimation  of  overall  uncer¬ 
tainties  for  integral  and  point  variables;  and  ( v)  order  of  accu¬ 
racy  and  Richardson  extrapolation. 

Step  (i)  is  self-explanatory.  For  steady  flow,  step  (ii)  con¬ 
sists  of  obtaining  estimates  for  iterative  and  grid  convergence 


uncertainties  (Usi  and  Usc^  respectively)  for  integral  and 
point  variables. 

The  estimates  for  grid  convergence  uncertainty  require  a  min¬ 
imum  of  three  grids  and  are  based  either  on  (a)  tiie  grid  conver¬ 
gence  metric 


(6,  -  62) 

^  (8) 

where  <f>  represents  either  an  integral  or  point  variable  with 
subscripts  1  and  2  corresponding  to  the  finer  and  coarser  grids, 
respectively,  or  (b)  the  grid  convergence  index  (Roache,  1997) 


GC/- 


36 

1) 


(9) 


where  r  is  the  grid  refinement  ratio  and  p  the  order  of  accuracy 
such  that  for  grid  doubling  and  second-order  methods  e  =  GCI. 
For  small  grid  refinement  and/or  order,  the  GCI  is  recom¬ 
mended  since  e  is  arbitrarily  small  and  inappropriate  as  a  metric 
of  ^d  convergence.  Note  that  for  small  (including  point 
variables  with  regions  of  small  <^),  c  should  be  normalized  by 
the  range  of  <^.  Decreasing  (increasing)  c/GCI  indicates  grid 
convergence  (div^ence)  with  uncertainty  estimates  based  on 
€  for  the  finest  grids.  Oscillatory  c/GCI  is  indeterminate,  with 
uncertainty  estimated  as  roughly  one-half  the  difference  be¬ 
tween  the  maximum  and  minimum  values.  For  simple  geome¬ 
tries  and  flows,  negligible  grid  convergence  uncertainty  is  at¬ 
tainable.  For  complex  geometries  and  flows,  convergence  may 
be  limited  and  oscillatory. 

The  estimates  for  iterative  convergence  uncertainty  are  based 
on  evaluation  of  the  iteration  records  for  both  integral  and  point 
variables.  TTie  level  of  iterative  convergence  is  determined  by 
the  number  of  orders  of  magnitude  reduction  and  magnitude  in 
the  residuals 


;  =  (10) 

where  n  is  the  iteration  number  and  can  either  be  the  solution 
variables  or  equation  imbalances  obtained  by  back  substitution. 
Average  (L2  norms)  or  maximum  values  are  used.  For  simple 
geometries  and  flows,  sixteen  orders  of  magnitude  reduction  of 
^  to  machine  zero  is  possible  such  that  the  iterative  convergence 
uncertainty  is  negligible.  However,  for  practical  geometries  and 
flows,  only  a  few  orders  of  magnitude  reduction  in  ^  to  about 


Table  2  Marine-propolsor  flow;  drcmnfercntial-average  mean-velodty  components  (radial-magnitude  average) 
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Rg-  Bib)  Downstream  {x/R  =  0^) 

Rg.  5  Marine-propulsor  fl0¥f:  circumferentially-averaged  mean  velocity 
components;  simulation  prediction  and  experimental  data;  comparison 
error  and  validation  uncertainty 


10“^  may  actuaDy  be  attainable.  In  this  case,  the  estimates  for 
iterative  convergence  uncertainty  are  based  on  statistics  of  the 
iteration  records  of  integral  and  point  variables  and  are  taken 
as  roughly  one-half  the  difference  between  the  maximum  and 
minimum  values. 

Step  (iii)  consists  of  determination  of  effects  of  exphcit  arti¬ 
ficial  dissipation  through  parametric  studies  with  uncertainty 
estimates  Usad  based  on  a  convergence  metric  similar  to  Eq. 
(8).  Step  (iv)  combines  the  uncertainties  (f/s/,  Usg^  Usad^  •  •  •) 
using  root  sum  square  (in  analogy  with  experimental  uncer¬ 
tainty  analysis  and  assumed  for  the  present  to  be  the  appropriate 
form)  to  obtain  overall  estimates  of  the  numerical  uncertainty 
UsN  for  both  integral  and  point  variables.  Step  (v)  consists  of 


obtaining  solutions  on  three  grids  in  the  asymptotic  range  (i.e., 
base,  halved,  and  doubled  grids  with  monotonically  decreasing 
c  at  rate  r^)  and  Taylor  series  expansions  to  determine  the  order 
of  accuracy  and  Richardson  extrapolated  coarse  grid  order  3p 
benchmarlo 

Steps  (i)-(iv)  are  required.  Step  (v)  is  useful  in  providing 
p  and  the  3p  benchmark  and  desirable  for  guiding  CFD  develop¬ 
ment,  but  often  difficult,  especially  in  obtaining  solutions  in  the 
asymptotic  range  for  complex  geometries.  Note  that  p  is  re¬ 
quired  for  evaluation  of  the  GCI,  that  other  evaluations  (e.g., 
rate  of  iterative  convergence)  are  also  useful  for  guiding  CFD 
development,  and  that  additional  uncertainty  estimates  may  be 
required  (e,g.,  local  and  global  non-conservation  of  mass,  mo¬ 
mentum,  energy,  etc.), 

Verification/Validation  for  a  Single  CFD  Code*  Vari¬ 
ables  chosen  to  illustrate  the  verification  and  validation  results 
are  the  propulsor  thrust  and  torque  performance  coefficients  {Kr 
and  Kq^  respectively ) ;  the  radiad  average  of  the  circumferential- 
average  mean-velocity  components  in  a  plane  upstream  (t/|, 
Vi,  Wi)  and  a  plane  downstream  ( f/2,  V2,  W2)  of  the  propeller; 
and  the  radial  profiles  of  the  circumferential-average  mean  ve¬ 
locities  in  the  same  planes. 

The  results  for  the  performance  coefficients  are  shown  in 
Table  1,  where  all  percentages  given  are  percentage  of  D.  The 
verification  results  show  that  the  uncertainty  in  grid  conver¬ 
gence  dominates  the  iteration  convergence  uncertainty,  resulting 
in  numerical  uncertainties  in  the  simulation,  Usn,  of  about  2.4 
percent.  The  uncertainties,  associated  with  the  experimen- 
taUy  determined  results  are  atout  2.0  percent,  leading  to  esti¬ 
mates  of  Uv  of  about  3.3  percent.  This  means  that,  for  this  flow 
condition,  validation  of  the  CFD  code  predictions  cannot  be 
achieved  better  than  about  3.3  percent  for  the  performance  coef¬ 
ficients.  Since  the  calculated  comparison  error  |  £|  is  less  than 
£/v,  validation  in  this  case  is  successful  at  the  3.3  percent  level 
for  both  Kt  and  Kq, 

The  results  for  the  average  velocities  in  the  upstream  and 
downstream  planes  are  shown  in  Table  2,  where  all  percentages 
given  are  percentage  of  the  uniform  inflow  velocity  (/q.  As  for 
the  performance  coefficients,  the  uncertainty  in  grid  conver¬ 
gence  is  always  greater  than  the  iteration  convergence  uncer- 
t^ty,  and  the  resulting  numerical  uncertainties  in  the  simula¬ 
tion,  UsN’^  are  0.6-0.9  percent.  The  uncertainties,  associated 
with  the  experimentally  determined  results  are  about  1.2  percent 
(upstream)  and  3.2  percent  (downstream),  leading  to  estimates 
of  Uv  of  about  1,3  percent  (upstream)  and  3.2  percent  (down¬ 
stream).  The  calculated  comparison  error  |F|  is  less  than  Uv 
for  average  velocities  £/i,  £/2»  V2,  and  1^2*  indicating  successful 
validation  for  those  variables.  Validation  of  the  simulations  of 
the  average  velocities  Vx  and  Wi,  however,  is  not  achieved.  It 
should  also  be  noted  that  since 

(where  T/  =  (t/,  V,  W)  and  Xj  =  r/R)  the  influence  of  the 
uncertainties  in  the  X^-locations  of  the  measurement  positions 
are  negligible. 


Table  3  Two-dimensioiial  turbulent  fiat-plate  boundary-layer  fiow:  streamwise  mean  velocity  (average  magnitude  across 
the  boundary  layer) 


Turbulence  model 

\E\% 

Uy% 

Ud% 

Ur% 

Us^% 

Us,% 

Uso% 

Ush/Ud 

|£|  +  Uv% 

Baldwin  Lomax  (1978) 

1.30 

2.52 

1.60 

1.60 

0.09 

1.95 

0.00 

1.95 

1.22 

3.82 

Chen  and  Patel  (1988) 

2.07 

2.40 

1.60 

1.60 

0.09 

1.79 

0.00 

1.79 

1.12 

4.47 

Myong  and  Kassagi  (1990) 

2.30 

2.67 

1.60 

1.60 

0.09 

2.14 

0.00 

2.14 

1.34 

4.97 

%  U,. 
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Tig.  6(a)  Simulation  prediction  and  experimental  data 


Fig.  6(c)  ]£|+Uv 


Fig.  B{b)  Comparison  error  and  validation  uncertainty 


Rg.6  Two-dimensionalturbulentflatplateboundarylayerflow-stream- 
wise  mean  velocity 


The  validation  comparisons  for  the  radial  profiles  of  the 
circumferential-average  velocity  components  are  shown  in 
Fig.  5(a)  for  the  upstream  plane  and  Fig.  5(^)  for  the 
downstream  plane.  In  this  presentation,  R  is  the  outer  radius 
of  the  propeller,  the  outer  radius  of  the  shaft  is  at  r/R  = 
0.2,  and  the  three  velocity  components  and  E  and  Uv  are 
normalized  by  Uo^  The  plots  of  U,  V,  and  W  versus  r/R 
show  the  comparisons  as  they  have  traditionally  been  made 
in  past  validation  efforts.  The  plots  of  E  and  ±Uv  present 
the  new  validation  view  introduced  in  this  article,  and  it  is 
immediately  obvious  where  validation  has  been  achieved 
and  where  it  has  not.  In  this  specific  case  presented,  valida¬ 
tion  is  achieved  for  some  variables  in  some  regions  of  the 
flow  but  not  for  all  variables  in  all  regions  of  the  flow  field. 
The  largest  errors  are  for  the  regions  of  the  flow  correspond¬ 
ing  to  the  shaft/blade  juncture  and  the  blade  tip  regions, 
which  is  confirmed  by  the  validation  results  using  the  com¬ 
plete  data  set  (Chen  and  Stem,  1997). 

Verification/ Validation  for  Comparison  of  Multiple  Tur¬ 
bulence  Models.  Variables  chosen  to  illustrate  the  verification 
and  validation  are  the  average  magnitude  across  the  boundary 
layer  of  the  streamwise  mean  velocity  and  the  profile  of  the 
streamwise  mean  velocity.  The  comparisons  are  for  three  turbu¬ 
lence  models  of  increasing  complexity,  i.e.,  the  Baldwin  and 
Lomax  (1978)  algebraic  model  (BL),  the  Chen  and  Patel 
(1988)  k-€  and  near-wall  model  (CP),  and  the  Myong  and 
Kasagi  ( 1990)  nonlinear  k-e  model  (MK).  In  the  presentations, 
the  velocity,  comparison  error,  and  validation  uncertainty  are 
normalized  by  the  edge  velocity  and  the  independent  variable 
Y  is  normalized  by  the  boundary-layer  thickness  6. 

The  results  for  tihie  average  velocity  are  shown  in  Table  3.  As 
for  the  previous  example,  the  uncertainty  in  grid  convergence 
dominates  the  iteration  convergence  uncertainty,  resulting  in 
numerical  uncertainties  Usn  for  the  three  models  ranging  from 
about  1.8--2.1  percent.  The  data  uncertainty  Uo  is  1.6  percent 
leading  to  validation  uncertainties  Uy  of  2.5-2.7  percent.  Here 
again,  the  contribution  to  Up  from  uncertainties  in  the  measure¬ 
ment  locations  (where  r,  =  U/Ue  and  Xj  =  Y/6)  is  negligible. 
All  three  models  are  validated  at  the  level  Uy  =  2.5 -2.7  percent. 
The  estimates  for  |  £|  +  Uy  range  for  increasing  model  com¬ 
plexity  from  3.8-5  percent. 

The  validation  comparisons  for  the  streamwise  mean  ve¬ 
locity  are  shown  in  Figs.  6(a-c).  The  simulation  prediction 
and  data  comparisons  U/Ue  vs.  Y/6  [Fig.  6(a)]  for  the  CP 
and  MK  models  show  relatively  large  underprediction  in  the 
mid-region  of  the  boundary-layer,  whereas  the  BL  model 
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shows  relatively  small  underprediction  in  the  near-wall  re¬ 
gion.  The  plots  of  E  and  ±Uv  show  that  the  BL  model  is 
validated  across  the  entire  boundary  layer,  whereas  the  CP 
and  MK  models  failure  is  confined  to  the  mid-region  of  the 
boundary  layer.  The  plots  of  the  |  £|  +  Uy  comparisons  for 
the  three  models  show  that  the  BL  model  maximum  values 
are  less  than  4  percent  except  for  the  near-wall  region 
where  the  values  increase  to  7  percent  Ue  at  the  wall,  whereas 
the  CP  and  MK  models  have  large  values  (8  percent  Ue)  at 
the  wall  and  in  the  outer  part  and  relatively  small  values  (3 
percent  l/«)  for  the  near-wall  region. 

5  Summary  and  Conclusions 

A  new  approach  to  CFD  validation  is  developed  that  gives 
proper  consideration  to  both  the  experimental  and  simulation 
uncertainties.  The  comparison  error  E  is  defined  as  the  differ¬ 
ence  between  the  data  D  (benchmark)  and  simulation  predic- 
tiou  value  S  and  thus  includes  the  errors  associated  with  the 
experimental  data  and  the  errors  associated  with  the  simula- 
tion.  The  validation  uncertainty  is  defined  as  the  combination 
of  the  uncertainties  in  the  experimental  data  and  the  portion 
of  the  uncertainties  in  the  CFD  prediction  that  can  be  esti¬ 
mated.  Estimates  for  Ussf  the  simulation  numerical  uncer¬ 
tainty,  are  obtained  through  verification  procedures  involving 
parametric,  convergence,  and  order-of-accuracy  studies.  The 
verification  procedures  are  discussed  in  detail.  Uo  includes 
contributions  from  the  independent  and  dependent  variable 
uncertainties  and  is  obtained  using  established  uncertainty 
analysis  procedures. 

/^eJkfaliaiE^njmceS^nty^  which  the^. 

validation  can  be^acEieved.  TTwcfiterion  fo^alidation  is  \ 
V4hat  I  £[  must  be  less  than  t/y.  T^e^co^tableleverof  valiSi^ 
tionirserby  pitipam  objecSves.  If  | £|  is  greater  than  I/y, 
than  validation  is  not  accomplished  and  the  magnitude  and 
sign  of  £  can  be  valuable  is  designing  strategies  to  improve 
the  mathematical  modeling.  When  a  validation  effort  in¬ 
volves  multiple  codes  and/or  models,  additionally  the  com¬ 
parisons  should  include  the  quantity  |£|  +  Uy,  Validation 
of  the  prediction  of  trends  involves  reduction  in  uncertainties, 
at  least  for  the  experiments,  through  inclusion  of  correlated 
bias  errors. 

Example  results  of  verification/ validation  are  presented  both 
for  a  single  CFD  code  and  for  a  comparison  of  multiple  turbu¬ 
lence  models.  A  RANS  CFD  code  is  used  for  two  applications 
for  which  numerical  and  experimental  uncertainty  analyses  are 
available:  marine-propulsor  flow  and  two-dimensional  turbulent 
flat-plate  boundary-layer  flow.  The  former  represents  a  practical 
geometry  and  is  used  for  the  single  CFD  code  validation  exam¬ 
ple,  whereas  the  later  represents  an  idealized  geometry  and  is 
used  for  the  comparison  of  turbulence  models  example.  The 
results  demonstrate  the  usefulness  of  the  proposed  validation 
strategy. 

The  authors  lecoimnend  the  adoption  of  this  approach  for 
CFD  code  validation.  It  will  be  useful  both  in  guiding  future 
develt^ments  in  CFD  through  validation  studies  and  in  the 
transition  of  CFD  codes  to  design  through  establishment  of 
credibility  and  ultimately  certification  once  procedures  for  the 
latter  are  established.  Realization  of  the  full  potential  of  the 
approach  requires  refinements  through  applications  using  vari¬ 
ous  CFD  methods  and  data  sets  (benchmarks),  especially  with 
regard  to  verification  procedures.  The  authors  also  recommend 
fliat  general  verification  procedures  be  established  similar  to 
fliose  used  here,  but  generalized  to  encompass  broad  categories 
of  simulation  methods  and  perhaps  additional  error  sources  in 
those  methods. 

Acknowledgments 

This  research  was  sponsored  by  the  Office  of  Naval  Re¬ 
search  under  grants  N00014-96-1-0018  and  N00014-97-1- 


(X)14  under  the  administration  of  Dr.  E.  P.  Rood.  The  authors 
gratefully  acknowledge  Dr.  Rood  and  other  colleagues  who 
made  significant  contributions  through  many  discussions.  In 
particular.  Prof.  W.  G.  Steele’s  comments  on  an  early  draft 
led  to  some  important  modifications.  Mr.  B.  Chen  and  Dr. 
M.  Sreedhar  generously  provided  the  example  results.  The 
criticisms  and  suggestions  of  the  six  reviewers  were  invalu¬ 
able  and  resulted  in  considerable  improvements  to  the  initial 
manuscript.  The  computations  were  performed  using  the 
NAVO  Oceanographic  Office  supercomputers. 

References 

American  Institute  of  Aeronautics  and  Astronautics  (AIAA),  1994,  *'Editoria] 
Policy  Statement  on  Numerical  Accuracy  and  Experimental  Uncertainty/*  AIAA 
Journal.  VoL  3^  No.  1. 

American  Institute  of  Aeronautics  and  Astronautics  (AIAA),  1997, 

* 'Guide  to  the  Assessment  of  Uncertainty  in  Computational  Fluid  Dynamics 
Simulations,**  AIAA  G-077,  Jan.  1997  draft  of  a  recommended  practices 
document. 

American  Society  of  Mechanical  Engineers  (ASME),  1993,  "Editorial 
Policy  Statement  on  the  Control  of  Numerical  Accuracy,**  ASME  Journal 
OF  Fluids  Engineering,  Vol.  1  IS,  No.  3. 

Baldwin,  B.  S.,  and  Lomax,  H.,  1978,  "Thin  Layer  Approximation  and 
Algebraic  Model  for  Separated  Turbulent  Flows,**  AIAA  Paper  AIAA  78- 
257. 

Beard,  R.  M.,  and  Landrum,  D.  B.,  1996,  "Effects  of  Kinetic  Rate  Uncer¬ 
tainty  on  the  Predicted  Performance  of  Small  Hydrogen  Thrusters,**  AIAA 
Paper  AIAA-96-2856. 

Blottner,  F.  G..  1990,  "Accurate  Navier-Stokes  Results  for  the  Hypersonic 
Flow  over  a  Spherical  Nosetip,**  Journal  of  Spacecraft  and  Rockets,  Vol. 
27,  No.  2. 

Brown,  K.  K.,  Coleman,  H.  W.,  and  Butas,  J.  P.,  1994,  "Impact  of  Uncer< 
tainty  on  Modeling  and  Testing  of  the  Space  Shuttle  Main  Engine,**  ASME 
FED-Vol.  205. 

Brown,  K.  K.,  Coleman,  H.  W.,  Steele,  W.  G.,  and  Taylor,  R.  P.,  1996, 
"Evaluation  of  Correlated  Bias  Error  Effects  in  Experimental  Uncertainty 
Analysis,**  AIAA  Journal.  Vol.  34,  No.  5. 

Celik,  I.,  Chen,  C.  J.,  Roache,  P.  J.,  and  Scheuerer,  G.,  eds.,  1993,  Quanti^ 
fication  of  Uncertainty  in  Computational  Fluid  Dynamics.  ASME  FED-Vol. 
158. 

Chen,  H.  C.,  and  Patel,  V.  C.,  1988,  "Near-Wall  Turbulence  Models  for 
Complex  Flows  Including  Separation,**  AIAA  Journal,  Vol.  26,  No.  6,  pp. 
641-648. 

Chen,  B.,  1996,  "Computational  Fluid  Dynamics  of  Four-Quadrant  Marine- 
Propulsor  Flow,**  M.S.  thesis.  Dept  of  Mechanical  Engineering,  University  of 
Iowa. 

Chen,  B.,  and  Stem,  F.,  1997,  "Computation  of  Marine-Propulsor  Flow — Part 
1:  Design  Condition,**  in  preparation. 

Coleman,  H.  W.,  1996,  "Uncertainty  Considerations  in  Validating  CFD  Codes 
with  Experimental  Data,**  AIAA  Paper  AlAA-96-2027. 

Coleman,  H.  W.,  and  Steele,  W.  G.,  1989,  Experimentation  and  Uncertainty 
Analysis  for  Engineers,  Wiley,  New  York. 

Coleman,  H.  W.,  and  Steele,  W.  G.,  1995,  "Engineeiing  Application  of  Uncer¬ 
tainty  Analysis,*’  AIAA  Journal,  Vol.  33,  No.  10. 

Coleman,  H.  W.,  Steele,  W.  G.,  and  Taylor,  R.  P.,  1995,  "Implications  of  Corre¬ 
lated  Bias  Uncertainties  in  Single  and  Comparative  Tests,**  ASME  Journal  of 
fluids  Engineering,  Vol.  117,  No.  4. 

Gresho,  P.  M.,  and  Taylor,  C.,  1994,  "Editorial,**  International  Journal  of 
Numerical  Methods  in  Fluids,  Vol.  19,  No.  12. 

International  Association  of  Hydraulic  Research  (lAHR),  1994,  "Guidelines 
for  Documenting  the  Validity  of  Corrqnitational  Moiling  Software.** 

International  Towing  Tank  Conference  (ITTC),  1996, 21st  ITTC  Proceedings, 
Bergen,  Norway. 

Jessup,  S.  D.,  1989,  "An  Experimental  Investigation  of  Viscous  Aspects  of 
Propeller  Blade  Flow,**  Ph.d.  Dissertation,  School  of  Engineering  and  Architec¬ 
ture,  The  C:atholic  University  of  America. 

Jessup,  S.  D.,  1994,  "Propeller  Blade  Flow  Measurements  Using  LDV,** 
ASME  Fluids  Engineering  Division  Summer  Meeting,  Lake  Tahoe,  NV. 

Longo,  J.,  Huang,  H.  P.,  and  Stem,  F.,  1998,  "Solid-Fluid  Juncture  Boundary 
Layer  and  Wake,’*  to  i^rpear  in  Experiments  in  Fluids. 

Marvin,  J.  G.,  1995,  "Perspective  on  Coit^tational  Fluid  Dynamics  Valida¬ 
tion,"  AIAA  Journal,  Vol  33,  No.  10. 

Mehta,  U.  B.,  1996,  "Guide  to  Credible  Computational  Fluid  Dynamics  Simu¬ 
lations,"  Journal  of  Propulsion  and  Power.  Vol.  12,  No.  5. 

Melnik,  R.  E.,  Siclari,  M.  J.,  Barber,  T.J.,  and  Verhoff,  A.,  "A  Process 
for  Industry  Certiheation  of  Physical  Simulation  Codes,"  AIAA  Paper  94- 
2235. 

Myong,  H.  K.,  and  Kasagi,  N.,  1991,  "Prediction  of  Anisotropy  of  the  Near- 
Wall  Turbulence  with  an  Anisotropic  Low-Reynolds  Number  k-e  Model,**  ASME 
Journal  of  fluids  Engineering,  Vol  IIZ 

Obericampjf,  W.  L.,  Blottner,  F.  G.,  and  Aeschliman,  D.  P.,  1995,  "Methodology 
for  Computational  Fluid  Dynamics  Code  Verification/Validation,"  AIAA  Paper 
AIAA-95-2226. 


802  /  Vol.  119,  DECEMBER  1997 


Transactions  of  the  ASME 


Roache,  P.J.,  1997,  “Quantification  of  Uncertainty  in  Computational  Fluid 
Dynamics,”  Annual  Review  of  Fluid  Mechanics,  Vol.  29,  pp.  123-160. 

Rood,  E.  P.,  1996,  “Validation  Strategy  for  “RANS”  Computational  Ship 
Hydrodynamics,”  Second  International  Conference  on  Hydrodynamics,  Hong 
Kong. 

Sreedbar,  M.,  and  Stem,  F.,  1997,  “Non-Linear  Eddy- Viscosity  Turbulence 
Model  for  Solid/Free-Surface  Boundary  Layer  and  Wake.”  3rd  Forum  on  Ad- 
yan^s  in  Num.  Modeling  of  Free  Surface  and  Interface  Fluid  Dynamics,  ASME 
FED  Summer  Meeting,  Vancouver,  Canada. 


Stem,  F..  Zhang,  D.  H.,  Chen,  B.,  Kim,  H.  T.,  and  Jessup,  S„  1994,  “Computa¬ 
tion  of  Viscous  Marine  Propulsor  Blade  and  Wake  Flow,”  Proc.  20th  ONR 
Symposium  on  Naval  Hydro,,  Santa  Barbara,  CA, 

Stem,  F.,  Paterson.  E.  G.,  and  Tahara,  Y.,  1996,  “CFDSHIP-IOWA:  Computa¬ 
tional  Fluid  Dynamics  Method  for  Surface-Ship  Boundary  Layers  and  Wakes  and 
Wave  Fields,”  Iowa  institute  of  Hydraulic  Research,  University  of  Iowa,  IIHR 
Report  No.  381. 

Zingg,  D.  W.,  1992,  “Grid  Studies  for  Thin-Layer  Navier-Stokes  Computations 
of  Airfoil  Rowfields,”  AIAA  Journal,  Vol.  30,  No.  10. 


Journal  of  Fluids  Engineering 


DECEMBER  1997,  Vol.  119/803 


Journal  of 
Fluids 

Engineering 


Discussion 


Uncertainties  and  CFD  Code  Validation^ 


Patrick  J.  Roache*^  The  paper  by  Coleman  and  Stem 
(1997)  proposes  a  new  approach  to  CH>  code  validation  that 
gives  consideration  to  both  experimental  and  simulation  uncer¬ 
tainties.  It  is  a  valuable  and  thought-provoking  contribution. 
Two  criticisms  follow. 

( 1 )  Hypothetical  data-simulation  comparisons  are  given  in 
Figure  1  (see  also  Coleman,  1996),  without  and  with  experi¬ 
mental  uncertainty  bars  (parts  a  and  b.  rcs^tively).  The  au¬ 
thors  state  that  one  might  interpret  part  a  to  indicate  that  Model 
1  is  superior  to  Model  2  since  it  seems  to  '^capture  the  trend 
of  the  data.**  In  contrast,  when  the  experimental  uncertainty  is 
considered  in  part  b,  *"it  is  obvious  that  arguing  for  one  method 
over  another  based  on  comparison  with  the  experimental  data 
is  wasted  effort  since  the  predictions  from  both  methods  fail 
well  within  the  data  uncertainty.’* 

This  negative  evaluation  would  be  true  if  and  only  if  the 
uncertainty  distribution  within  the  plotted  range  of  experimental 
uncertainty  (commonly  called  “error  bars’* )  was  flat,  i.e.  a  step 
fimction.  (For  example,  if  the  “error  bar’*  covered  the  data 
value  D  to  ±  10%,  this  flat  distribution  would  give  equal  proba¬ 
bility  that  the  true  value  was  at  Z)  + 1%,  D  -3%,  D  +9%,  etc. 
or  at  D  itself.)  In  such  a  case,  our  probabilistic  knowledge  is 
indifferent  to  any  values  within  the  range  of  the  “error  bar’* 
and  therefore  to  the  relative  merits  of  Model  1  and  2,  and  the 
authors*  argument  holds.  Of  course,  we  do  not  know  what  the 
appropriate  probability  distribution  is,  but  most  physical  experi¬ 
ments  have  some  preference  for  the  data  values  reported,  and 
the  default  model  of  probability  is  a  Gaussian  distribution.  If, 
in  the  judgment  of  the  reader,  this  is  the  more  likely  distribution, 
then  there  is  probably  a  trend  in  the  true  values,  and  Model  1 
does  indeed  “capture  the  trend  of  the  data.”  One  could  readily 
express  this  quantitatively  as  a  weighted  integral  over  the  as¬ 
sumed  probability  distributions  within  the  “error  bars,’*  but 
qualitatively,  I  believe  that  the  naive  preference  for  Model  I  is 
justifiable. 

(2)  The  principal  thrust  of  the  paper  is  that  the  validation 
uncertainty  Uv  should  be  the  key  metric  in  the  validation  pro¬ 
cess.  They  define  the  “validation  uncertainty  Uy  as  the  combi¬ 
nation  of  all  uncertainties  that  we  know  how  to  estimate,’* 
which  includes  experimental  and  numerical  uncertainties,  the 
latter  including  lack  of  grid  convergence.  With  the  *  ’comparison 
error’’  E  defined  as  the  difference  between  the  experimental 
Hntfl  set  value  the  value  produced  by  the  simulation,  they  then 
interpret  the  condition  |  E|  <Vy  to  be  the  criteria  for  claiming 
that  validation  has  been  achieved  at  the  Uy  level. 


'  By  H.  W.  Coleman  and  F.  Stem  published  in  the  December  1997  issue  of  the 
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The  authors  are  well  aware  of  a  paradox  inherent  in  this 
criterion,  that  increasing  uncertainty  of  the  experiments  or 
the  simulation  (e.g.,  less  convincing  grid  convergence  tests) 
makes  it  easier  to  achieve  validation.  (The  resolution  of  the 
paradox  lies  in  the  recognition  that  the  validation  level  so 
achieved  is  less  demanding,  i.e.  Uy  is  greater.)  See  more 
complete  discussion  in  Coleman  and  Stem  (1997)  and  in 
Roache  (1998).  However,  besides  the  evident  potential  for 
misinterpretation  in  the  use  of  Uy^  a  more  basic  problem 
exists  with  their  proposal;  it  fails  to  account  for  an  acceptable 
error  tolerance  in  the  Validation.  This  is  easily  demonstrated 
by  the  following  “thought  experiment.’* 

Imagine  a  programmatic  need  for  Validation  with  a  5%  error 
tolerance  (call  it  TOLy),  That  is,  the  need  is  for  a  model/code 
that  can  predict  physical  reality  to  within  5%.  Further  imagine 
the  limit  of  good  experiments  and  calculations,  i.e.  vanishingly 
small  experimental  uncertainty  and  simulation  uncertainty,  so 
that  their  validation  uncertainty  Uy  is  also  vanishingly  small. 
Finally,  imagine  excellent  agreement  between  experiment  and 
simulation,  say  an  observation  error  \E\  =0.1  percent.  Then, 
their  proposed  criterion  does  not  allow  this  to  be  considered  a 
validation.  No  matter  how  good  the  agreement  between  experi¬ 
ment  and  simulation,  nor  how  lax  the  programmatic  iieeds,  if 
the  agreement  is  not  as  good  as  the  combined  uncertainties  from 
experiment  and  simulation,  it  cannot  be  considered  a  validation, 
in  this  approach. 

A  mo^fleation  of  their  validation  criterion  to  include  a  pro¬ 
grammatic  validation  tolerance  TOLy  would  avoid  this  diffi¬ 
culty. 

|£|  <  C/v+  TOLy 

However,  the  previously  mentioned  paradox  remains,  e.g.  in¬ 
creasing  experimental  and/or  numerical  uncertainty  still  make 
it  easier  to  achieve  validation,  at  the  level  now  defined  by  Uy 
+  TOLy,  Further  consideration  by  the  simulation  community 
will  be  required  to  determine  whether  or  not  this  modification 
makes  their  proposal  practical.  See  also  further  discussion  in 
Roache  (1998). 
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instance,  the  uncertainty  bands  plotted  included  a  significant 


Journal  of  Fluids  Engineering 


Copyright  €>  1998  by  ASME 


SEPTEMBER  1998,  Vol.  120  /  635 


random  uncertainty  component  and  the  data  points  were  single 
realizations,  we  would  hesitate  to  bet  that  the  trend  (curvature) 
shown  would  be  the  same  as  that  shown  by  a  succeeding  set 
of  single  realizations  of  if  many  data  sets  were  obtained  and 
the  average  values  were  plotted.  On  the  other  hand,  in  the  very 
special  case  that  ( 1 )  we  knew  each  data  point  was  affected  by 
the  same  systematic  uncertainty  that  did  not  vary  with  X  and 
(2)  the  data  points  were  averages  of  many  readings  so  that  the 
random  component  of  uncertainty  was  negligible,  we  would 
believe  the  trend  shown. 

His  comments  on  the  second  point  are  insightful  and  cover 
a  case  we  did  not  consider  or  discuss  in  the  article.  His  “error 
tolerance**  corresponds  to  our  “required  level  of  validation*' 
discussed  in  the  article.  Call  this  since  validation  is  re¬ 
quired  at  that  uncertainty  level  or  below.  The  salient  point  raised 
by  Roache  is  how  should  we  interpret  the  case  Uv  <  |  £  |  < 
C/fcqd?  We  believe  that  case  should  also  be  considered  a  success¬ 
ful  validation  since  the  comparison  error  is  less  than  f/nqd  and 
the  noise  level  Uv  allows  discrimination. 

From  a  general  perspective,  if  we  consider  the  three  variables 
f/v*  |£|  and  there  arc  six  combinations  (assuming  none 
of  the  three  variables  are  equal): 


1.  |£|  <Uy<U^ 

2.  \E\<U^<Uv 

3.  Ur^<  \E\  <Uy 

4.  Uy<  |£|  < 

5.  Uy<U^<  \E\ 

6.  Ur^<Uv<  \E\ 

In  cases  1,  2,  and  3,  |  £  |  <  Uv\  validation  is  achieved  at  the 
Uy  level;  and  the  comparison  error  is  below  the  noise  level,  so 
attempting  to  decrease  the  error  (£sma)  due  to  the  modeling 
assumptions  in  the  simulation  is  not  feasible  from  an  uncertainty 
standpoint.  In  case  1  validation  has  been  achieved  at  a  level 
below  C/feqdt  so  validation  is  successful  from  a  programmatic 
standpoint. 

In  cases  4,  5,  and  6,  t/y  <  I  £  | ,  so  the  comparison  error  is 
above  the  noise  level  and  attempting  to  decrease  £sma  is  feasible 
from  an  uncertainty  standpoint.  In  case  4  (the  one  pointed  out 
by  Roache),  validation  is  successful  at  the  |£j  level  from  a 
programmatic  standpoint.  Note  that  for  Uy  |£|,  £  corre¬ 
sponds  to  £sma  and  the  error  from  the  modeling  assumptions 
can  be  determined  unambiguously. 

We  did  not  follow  the  logic  in  comparing  | £|  with  (f/y  + 
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ABSTRACT 

Results  are  presented  from  overlapping  towing 
tank  tests  between  three  institutes  for  resistance, 
sinkage  and  trim,  wave  profiles  and  elevations,  and 
nominal  wake  using  the  same  model  geometry  and 
conditions,  including  rigorous  application  of  standard 
uncertainty  assessment  procedures.  Two  of  the 
institutes  used  5.7  m  models  whereas  the  third  institute 
used  a  smaller  3  m  model.  Comparison  variables  were 
defined  for  data-reduction  equations  and  data 
differences  and  data-difference  uncertainties.  Detailed 
descriptions  were  provided  of  facilitiesv  measurement 
systems,  data-acquisition  and  reduction  procedures,  and 
uncertainty  assessment.  Results  were  discussed  with 
regard  to  levels  and  causes  of  data  differences  and  data- 
difference  uncertainties  and  to  estimate  facility/model 
geometry  and  scale  effect  biases.  For  same  size  5.7  m 
models,  data  differences  were  in  general  oscillatory, 
and  in  many  cases,  larger  in  magnitude  than  data- 
difference  uncertainties,  which  indicates  unaccounted 
for  bias  and  precision  limits  and  that  current  individual 
facility  uncertainty  estimates  are  often  too  optimistic. 
Scale  effects  for  the  3  m  model  are  only  evident  for 
resistance  and  trim  tests  at  high  Fr.  Facility/model 
geometry  and  scale  effect  bias  are  estimated  based  on 
comparisons.  Uncertainty  estimates  including  such 
biases  may  provide  better  estimates,  especially  for  use 
in  CFD  validation,  which  is  the  recommendation  of  the 
present  study  along  with  efforts  towards  improvement 
of  individual  institute  uncertainty  estimates.  Use  of 
standard  models  and  current  ITTC  efforts  in  providing 
standard  quality  manual  procedures  for  towing  tank 
tests  and  uncertainty  estimates  will  also  be  helpful  in 
this  regard. 

INTRODUCTION 

Towing  tank  testing  is  undergoing  change 
from  routine  tests  for  global  variables  to  detailed  tests 
for  local  variables  for  model  development  and 


computational  fluid  dynamics  (CFD)  validation,  as 
design  methodology  changes  from  model  testing  and 
theory  to  simulation-based  design.  Such  detailed 
testing  requires  that  towing  tanks  utilize  advanced 
modem  instrumentation  with  complete  documentation 
of  test  conditions,  procedures,  and  uncertainty 
assessment.  The  requirements  for  levels  of 
uncertainties  are  even  more  stringent  than  those 
required  previously  since  they  are  a  limiting  factor  in 
establishing  the  level  of  validation  and  credibility  of 
simulation  technology.  Also,  routine  test  data  is  more 
likely  utilized  in  house,  whereas  detailed  test  data  is 
more  likely  utilized  internationally,  which  additionally 
requires  use  of  standard  procedures  and  establishment 
of  benclunaik  levels  of  data  uncertainties.  Det^ed 
testing  offers  new  opportunities  for  towing  tanks,  as  the 
amount  and  complexity  of  testing  is  increased. 
International  collaborations  are  attractive  from  a 
resource  perspective. 

The  benchmark  database  for  CFD  validation 
for  resistance  and  propulsion  is  fairly  extensive  with 
current  focus  on  modem  hull  forms  and  detailed  tests  as 
reported  by  the  Resistance  Committee  of  the  22“* 
International  Towing  Tank  Conference  (ITTC,  1999). 
Tanker  (KVLCC2),  container  ship  (KCS),  and  surface 
combatant  (DTMB  5415)  hull  forms  were 
recommended  for  use  and  are  currently  being  used  as 
test  cases  in  the  Gothenburg  2000  Workshop  on  CFD 
for  Ship  Hydrodynamics  (Gothenburg  2000; 
http://www.iihr.uiowa!edii/gothenburg2000/). 

KVLCC2  and  KCS  were  conceived  by  the  Korean 
Institute  of  Ships  &  Ocean  Engineering  (KRISO) 
specifically  as  test  cases  for  modem  taidccr  and 
container  ship  hull  forms  for  CFD  validation  for  ship 
hydrodynamics  ca.  1997.  KVLCC2  and  KCS  have 
bulbous  bows  and  bulbous  cruiser  and  transom  stems, 
respectively.  The  KVLCC2  and  KCS  data  were 
procured  by  KRISO  (Van  et  al.,  1997  and  1998a,  b)  in 
collaboration  with  Pohang  University  of  Science  and 
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Technology,  Korea  (Lee  et  al.,  1998)  and  Ship 
Research  Institute,  Japan,  respectively.  DTMB  5415 
was  conceived  by  David  Taylor  Model  Basin,  USA 
Navy  as  a  preliminary  design  for  a  surface  combatant 
ca.  1980  with  a  sonar  dome  bow  and  transom  stem. 
The  DTMB  5415  data  were  procured  by  DTMB 
(Ratcliffe,  1995;  http://www50.dt.navv.mil/541 5A  in 
collaboration  with  Istituto  Nazionale  per  Studi  ed 
Esperienze  di  Architettura  Navale  (INSEAN),  Italy 
(Avanzini  et  al,,  1998,  Avanzini  et  al.,  2000,  Olivieri 
and  Penna,  1999,  Olivieri  and  Penna,  2000),  and  Iowa 
Institute  of  Hydraulic  Research,  USA  (Longo  and 
Stem,  1998,  Gui  et  al.,  1999,  Longo  et  al.,  2000,  Stem, 
2000:  http://www.iihr.uiowa.edu/-^owtank/). 

The  present  paper  describes  the  international 
collaboration  on  DTMB  model  5415  between  DTMB, 
INSEAN,  and  IIHR.  The  collaboration  was  done  at 
DTMB  using  5415  (5.72  m,  1/24.8  scale  model),  at 
INSEAN  using  INSEAN  model  2340  and  2340A  (exact 
gcosyms  of  5415),  and  at  UHR  using  DTMB  model 
5512  (3.038  m,  1/46.6  scale  geosyra  of  5415).  Figure  1 
shows  models  5415,  2340A,  and  5512.  Between  all 
three  institutions  many  conditions  and  physics  are 
under  investigation.  The  conditions  include  bare  hull 
without  (all)  and  with  ^pendages  and  propulsor 
(DTMB),  and  fixed  and  free  model  (all).  The  physics 
are  comprehensive  and  include  model  size  (IIHR), 
facility  biases  (all),  Reynolds  number  (Re)  effects  (all), 
boundary  layer  and  wake  (INSEAN),  stem  flow  (all), 
Froude  number  (Fr)  effects  (all),  bow  and  transom  flow 
(DTMB),  wave  breaking  (INSEAN),  and  turbulence 
and  head  waves  (IIHR).  The  uncertainty  assessment 
procedures  closely  follow  ITTC  (1999) 
recommendations.  Overlapping  tests  for  resistance, 
sinkage  and  trim,  wave  profile,  wave  elevations,  and 
nominal  wake  are  included  for  evaluation  between 
institutes  of  facilities;  measurement  systems;  test 
procedures;  uncertainty  assessments;  model  size, 
offsets,  and  turbulence  stimulation;  and  facility/model 
geometry  and  scale  effect  biases,  through  comparisons 
of  both  data  and  uncertainties. 

The  results  of  the  overlapping  tests  build  on 
information  provided  by  the  Cooperative  Experimental 
Program  of  the  Resistance  Committees  of  the  17-19 
me  and  the  cooperative  uncertainty  assessment 
example  for  resistance  test  of  the  Resistance  Committee 
of  the  22nd  ITTC.  The  former  provided  comparison 
between  up  to  22  institutes  of  global  (resistance, 
sinkage  and  trim,  wave  profile,  wave  cut,  wake  survey, 
form  factor,  and  blockage)  and  local  (surface  pressure 
and  boundary  layer  traverses)  data  for  a  standard 
geometry  (Series  60)  of  different  sizes  (1.2-9.6  m),  but 
did  not  consider  uncertainty  assessment  The  latter 
built  on  the  former  in  providing  comparison  between  7 


institutes  of  resistance  test  uncertainties  following 
standard  uncertainty  assessment  methodology,  but  for 
different  model  geometries  and  sizes  (Series  60, 
container  ships,  and  5415).  Present  work  builds  on 
both  in  providing  comparisons  between  3  institutes  of 
both  data  and  uncertainties  for  the  same  model 
geometry  of  2  sizes  (3  and  5.72  m).  Such  comparisons 
between  facilities  is  apparently  relatively  uncommon  in 
other  fields  such  as  aerospace  and  mechanical 
engineering,  which  may  be  due  to  increased  complexity 
of  routine  ship  model  testing  due  to  viscous  and  fi^e 
surface  effects  in  comparison  to  routine  testing  in  other 
fields.  The  results  are  timely  with  regard  to  the 
Gothenburg  2000  Workshop  on  CFD  for  Ship 
Hydrodynamics  and  should  be  taken  into  consideration 
in  reaching  conclusions  regarding  levels  of  CFD 
validation. 

The  focus  herein  is  on  the  overlapping  tests; 
however,  highlights  are  given  of  the  overall  test 
program.  Sections  describe,  respectively:  the 
overlapping  test  design,  comparison  variables,  and 
conditions;  facilities,  measurement  systems,  and 
procedures;  uncertainty  assessments;  CFD 
validation/complementary  CFD;  comparisons  of  results; 
highlights  of  the  overall  test  program;  and  conclusions. 


OVERLAPPING  TEST  DESIGN,  COMPARISON 
VARIABLES,  AND  CONDITIONS 

The  most  typical  towing-tank  tests  were 
selected  for  the  overlapping  tests,  i.e.,  resistance, 
sinkage  and  trim,  wave  profile,  wave  elevations,  and 
nomi^  wake.  Each  institute  followed  their  usual 
procedures;  however,  special  consideration  was  given 
to  integration  of  uncertainty  assessment  into  all  phases 
of  the  experimental  process,  CFD  validation,  and 
complementary  CFD. 

Comparison  variables  were  defined  for  total 
and  residuary  Cr  resistance,  sinkage  a  and  trim 
t,  wave  profile  jmd  elevations  and  nominal  wake 
mean  velocity  V  and  pressure  Cp,  as  given  by  the 
following  data-reduction  equations: 

(la) 

Cn  =  C^  -cf'(I+k)  (lb) 
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cj^Sdeg  sgjgcted  as  the  comparison  variable 
for  resistance,  since  it  calibrates  all  data  to  the  same 
temperature  T=15®C;  thereby,  enabling  quantitative 
comparisons  for  similar  tests  and  models  at  different 
institutes,  as  recommended  by  ITTC  Quality  Manual 
Procedure  4.9-03-03-01.2  1978  ITTC  Performance 
Prediction  Method  k  is  the  form  factor  and  Cp  is  the 
flat-plate  friction  line,  a  and  T  are  defined  without  Fr^ 
in  the  denominator.  V ,  and  Cp  are  normalized  by 


model  length  L,  carriage  speed  Uc,  and  dynamic 
pressure  pC//, respectively. 

To  facilitate  the  comparisons  of  data  procured 
at  the  different  institutes,  data  (A,  B,  C),  data 
differences  Dj,  and  data  differences  uncertainty 
U 2^  variables  are  defined  as  follows: 

(A,B.C) = ,  a.x.q.y.  Cp  )i . 
i  =  A(DTMB  ),  B(INSEAN ),  C(IIHR  ) 

(7) 

^AB  - 

(8) 

(9) 

Dac^A-C 

(10) 

(11) 

b^bc  -  B-C 

(12) 

(13) 

Data  differences  Di  are  attributed  to  differences  in 
facility  (size,  water  quality,  carriage);  model  size  (for 
C),  of&ets,  and  turbulence  stimulation;  and 


measurement  systems  and  procedures.  Comparison  of 
two  institutes  is  relatively  straightforward.  For 

|Z),|st/fl^.,i=AB,AC,BC  (14) 

data  between  institutes  agree  at  level 
Presumably,  design  sets  the  required  level  of  agreement 
between  institutes.  Better  agreement  requires  reduaion 
of  t//).  and  possibly  Dj.  For 

lAl>i^£t.i=AB.AC,BC  (15) 

data  disagreement  is  attributed  to  model  size  (for  C) 
and  unaccounted  for  bias  B(x,b,C)  precision 
P(A,B,C)  Comparison  of  three  institutes  is  not 

straight  forward  as  there  are  many  combinations  of 
equation  (14)  and  (15).  Various  combinations  were 
considered  such  as  A=B-2C,  A+C-2B,  and  B-hC-2A; 
however,  it  was  difficult  to  draw  conclusions  from 
these  results.  Presentations  of  comparisons  are 
therefore  limited  to  (A,  B,  C),  A,  U(A,B.ch  and  U q.  , 

The  conditions  at  each  institute  for  the 
overlapping  tests  are  summarized  in  Table  1.  For  each 
test,  the  measurement  system,  Fr,  Re,  average 
temperature  Tavc.  density,  kinematic  viscosity,  surface 
tension,  density  of  data,  and  model  installation  with 
displacements  at  bow  (FP)  and  stem  (AP),  is  indicated. 

FACILITIES,  MEASUREMENT  SYSTEMS,  AND 
PROCEDURES 

Facilities.  Experiments  with  5415  arc 
performed  in  basin  no.  2  (575  m  long,  15.5  m  wide,  6.7 
m  deep).  Basin  no.  2  is  equipped  with  an  electro- 
hydraulically  operated  drive  carriage  and  capable  of 
speeds  of  10.3  m/s.  Sidewall  and  endwall  beaches 
enable  20-minute  intervals  between  carriage  mns. 
Towing-tank  water  is  supplied  by  the  Washington 
Suburban  Sanitation  Commission,  Experiments  with 
2340A  are  performed  in  towing  tank  no.  2  (220  m  long, 

9  m  wide,  3.6  m  deep).  Towing  tank  no.  2  is  equipped 
with  a  single  drive  carriage  that  is  capable  of  speeds  of 

10  m/s.  Sidewall  and  endwall  beaches  enable  20- 
minute  intervals  between  carriage  runs.  Towing  tank 
water  is  spring  water.  Experiments  with  5512  are 
performed  in  the  IlHR  towing  tank  (100  m  long  and 
3.048  m  wide  and  deep).  The  IIHR  tank  is  equipped 
with  an  electric-motor  operated  drive  carriage  that  is 
cable  driven  by  a  15-horsepower  motor  and  capable  of 
speeds  of  3  m/s.  Sidewall  and  endwall  beaches  enable 
twelve-minute  intervals  between  carriage  runs. 
Towing-tank  water  is  supplied  by  the  city  of  Iowa  City. 

Model  geometry.  5415  was  constructed  at  the 
DTMB  model  workshop  in  1980  firom  a  blank  of 
laminated  wood  and  a  con^)uterized  numerical^cutting 
(CNC)  machine.  Turbulence  stimulation  is  at  x=O.05 
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with  cylindrical  studs  having  3.2  mm  diameter,  2.5  mm 
height,  and  spaced  5.0  ram.  The  geometry  offset 
measurement  system  consists  of  twenty-five  wooden 
templates. 


(a)  DTMB  model  5415 


(c)nHR  model  5512 

Fig.  1:  Model  geometries  for  the  overlapping  tests. 


2340A  was  constructed  in  1998  at  the  INSEAN  model 
woricshop  from  a  blank  of  laminated  wood  and  a  CNC 
machine.  Turbulence  stimulation  is  at  x=K),01  with 
cylindrical  studs  having  3.0  mm  diameter,  3.0  mm 
height,  and  spaced  30.0  mm.  The  geometry  offset 
measurement  system  consists  of  computer-aided  design 
(CAD),  hand-cut  templates,  level  table,  right  angle, 
plumb,  and  rulers  and  feeler  gauges.  The  data  is 
reduced  by  computing  crossplane  and  global  average 
values  for  the  error  in  the  offsets  for  each  coordinate 
and  for  S.  5512  was  constructed  in  1996  at  the  DTMB 
model  workshop  from  molded  fiber-reinforced 
Plexiglas.  Turbulence  stimulation  is  at  x=0.05  with 
cylindrical  studs  having  3.2  mm  diameter,  1.6  mm 
height,  and  10.0  mm  spacing.  The  measurement  system 
and  data-reduction  procedures  for  determination  of  the 
errors  in  the  geometry  offsets  and  S  are  virtually  the 
same  as  at  INSEAN,  however,  the  IIHR  templates  are 
CNC  milled. 

Carriage  speed.  At  each  facility,  carriage 
speed  Uc  is  measured  with  encoder-based  measurement 
systems  and  PC  data  acquisition.  The  operating 
principle  is  integer  pulse  (n)  counting  at  a  wheel- 
mounted  (diameter=D)  encoder  in  known  time  intervals 
(At).  The  data-reduction  equations  are  of  the  form 


(16) 


where  a=(520,  1000,  8000),  respectively  for  (A,  B,  C). 
Pulse  count  data  is  either  passed  to  a  PC  directly  as  a 
ftequency  of  through  an  analog-to-dighal  (AD) 
conversion.  DTMB  utilizes  a  520-pulse,  magnetic 
encoder  carriage  speed  measurement  system  that  is 
calibrated  periodically  to  monitor  its  accuracy.  Linear 


resolution  is  0,33  mm/pulse.  Data  acquisition  is  done 
through  collection  of  2000  discrete  samples  over  5 
seconds  at  400  Hz.  Pulses  are  counted  in  At  and 
entered  into  the  PC  as  a  frequency.  Data  reduction  is 
completed  through  statistical  analysis  of  the  sample 
population  (average,  standard  deviation,  minimums, 
maximums,  outliers).  Outliers  arc  identified  and 
deleted  using  Chauvenet’s  criterion.  INSEAN  utilizes  a 
1000-window,  optical  encoder  carriage  speed 
measurement  system.  Linear  resolution  is  1 

mm/window.  Data  acquisition  is  done  through  a  16-bit 
AD  card  by  collection  of  300  smnples  over  10  secoiKls 
at  30  Hz.  Digital  output  from  the  encoder  is  converted 
to  velocity  prior  to  being  recorded  by  the  PC  AD  card 
and/or  digital  recorder  (DAT).  Data  reduction  is 
similar  as  for  DTMB.  IIHR  utilizes  an  8000-window, 
optical  encoder  carriage  speed  measurement  system  that 
is  calibrated  periodically  at  IIHR  to  determine  its 
voltage-frequency  relationship.  Linear  resolution  is 
0.15  mm/wittdow.  Data  acquisition  is  done  througlr 
collection  of  2000  samples  over  10  seconds  at  200  Hz. 
Pulses  undergo  two  conversions  (DA  and  AD)  before 
entering  the  PC.  The  data-reduction  processes  are 
similar  as  for  DTMB  and  INSEAN. 

For  the  resistance  and  sinkage  and  trim  tests, 
the  models  are  free  to  heave,  pitch,  and  roll  but 
restrained  in  surge,  sway,  and  yaw.  For  all  other  tests, 
the  models  are  restrained  from  heave,  pitch,  roll,  surge, 
sway,  and  yaw  but  fixed  at  their  dynamic  sinkage  and 
trim  (Table  1). 

Resistance*  Resistance  Ctisc  is  measured 
with  loadcell-based  measurement  systems  and  PC  data 
acquisition.  The  towing  force  in  (kg)  is  converted  to 
Newtons  (N)  by  multiplication  with  g  (gA=9,8009  m/s?; 
gB*=9.8033  m/^;  gc*9.8031  m/s^)  based  on  the  local 
latitude  (Halliday  and  Resnick,  1981).  Towing  tank 
water  temperature  is  measured  daily  at  the  model  mid 
draft  using  a  digital  thermometer  from  which  density  p 
and  kinematic  viscosity  v  are  linearly  interpolated  using 
fresh  water  values  as  recommended  by  ITTC  Quality 
Manual  Procedure  4.9-03-01-03  Density  and  Viscosity 
of  Water.  The  form  factor  k==0.15  is  calculated  using 
Prohaska’s  method,  as  recommended  by  ITTC  Quality 
Manual  Procedure  4.9-03-03-01.2  1978  ITTC 

Performance  Prediction  Method.  Data-reduction 
processes  are  the  same  as  for  the  carriage-speed  tests. 
DTMB  uses  a  variable  reluctance,  in-house 
manufactured  loadcell,  signal  conditioner,  and  16-bit 
AD  card  with  carriage  PC  for  the  resistance  tests.  The 
loadcell,  signal  conditioner,  and  carriage  PC  AD  card 
are  statically  calibrated  on  a  DTMB  test  stand  to 
determine  the  voltage-mass  relationship.  Data 
acquisition  is  done  through  collection  of  2000  discrete 
samples  over  5  seconds  at  400  Hz.  Data  is  filtered 
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through  a  10  Hz  low-pass  filter.  INSEAN  uses  a 
Hottinger  Baldwin  Messtechnik  model  Ul,  50  kg 
loadcell,  signal  conditioner,  and  16-bit  AD  card  with 
PC  for  the  resistance  tests.  The  loadcell,  signal 
conditioner,  and  carriage  PC  AD  card  are  statically 
calibrated  on  a  Kempf  and  Remmers  precision  test 
stand  to  determine  the  voltage-mass  relationship.  Data 
acquisition  is  done  through  collection  of  300  discrete 
samples  over  10  seconds  at  30  Hz.  Amplified  analog 
voltages  are  converted  to  frequency  (3000±2500Hz)  for 
transmission  to  the  AD  card  to  reduce  signal  sensitivity 
to  noise.  Data  is  filtered  through  a  10  Hz  low-pass 
filter.  IIHR  uses  a  Nisshio  strain-gage  type  20  kg 
loadcell,  signal  conditioner,  and  12-bit  AD  card  with 
PC  for  the  resistance  tests.  The  loadcell,  signal 
conditioner,  and  carriage  PC  AD  card  are  statically 
calibrated  on  an  llHR  test  stand  to  determine  the 
voltage-mass  relationship.  Data  acquisition  is  done 
through  collection  of  2000  discrete  samples  over  10 
seconds  at  200  Hz.  Data  is  filtered  through  a  3  Hz  low- 
pass  filter. 

Sinkage  and  trim.  Sinkage  a  and  trim  t  are 
measured  with  potentiometer-based  measurement 
systems  and  PC  data  acquisition.  The  potentiometers 
sense  displacements  of  die  model  at  the  FP  and  AP 
which  are  converted  to  c  and  T  with  equations  (2)  and 
(3).  Data-reduction  processes  are  the  same  as  for  the 
resistance  tests.  DTMB  employs  linear  potentiometers, 
signal  conditioners,  and  a  16-bit  AD  card  with  PC  for 
the  sinkage  and  trim  tests.  The  potentiometers,  signal 
conditioners,  and  carriage  PC  AD  card  are  statically 
calibrated  on  a  DTMB  test  stand  to  determine  the 
voltage-displacement  relationship.  Data  acquisition  is 
done  through  collection  of  500  discrete  samples  over  5 
seconds  at  100  Hz.  Data  is  filtered  through  a  10  Hz 
low-pass  filter.  INSEAN  employs  rotative 
potentiometers,  signal  conditioners,  and  a  16-bit  AD 
card  with  PC  for  the  sinkage  and  trim  tests. 
Displacement  measurements  are  made  by  conversion  of 
vertical  to  angular  displacements  through  weight- 
balanced,  mechanical  parallelograms.  The 
potentiometers,  signal  conditioners,  and  carriage  PC 
AD  card  are  statically  calibrated  on  an  INSEAN  test 
stand  to  determine  the  voltage-displacement 
relationship.  Data  acquisition  is  done  through 
collection  of  300  discrete  samples  over  10  seconds  at 
30  Hz.  Data  is  filtered  through  a  10  Hz  low-pass  filter. 
IIHR  employs  linear  potentiometers  and  a  12-bit  AD 
card  with  PC  for  the  sinkage  and  trim  tests.  The 
potentiometers  and  carriage  PC  AD  card  are  statically 
calibrated  on  an  IIHR  test  stand  to  determine  the 
voltage-displacement  relationship.  Data  acquisition  is 
done  throu^  collection  of  2000  discrete  samples  over 
10  seconds  at  200  Hz.  Data  is  not  filtered. 


Wave  profile.  Wave  profiles  ^  are  measured 
with  different  measurement  systems  and  normalized 
with  model  length  L.  DTMB  utilizes  a  waterproof 
pencil  and  hull-based  grid  system  for  the  wave  profile 
tests.  Wave  profiles  are  marked  on  the  model  as  it  is 
towed  throu^  the  basin.  Vernier  calipers  are  used  to 
quantify  the  wave  profile  heights  referenced  to  the  full¬ 
load  water  line.  INSEAN  utilizes  a  photogr^hic  and 
digitizing  measurement  system  with  a  hull-based  grid 
system  for  the  wave  profile  tests.  Data  acquisition  is 
done  by  photographing  the  wave  profile  in  sections 
(20%  L)  and  digitizing  the  negatives  with  a  hi^- 
resolution  scanner.  Wave  heights  are  quantified  at  x- 
stations  on  the  model  with  CAD  software.  IIHR 
utilizes  adhesive  maricers,  flexible  ruler,  level  table, 
height  gauge,  and  hull-based  grid  system  for  the  wave 
profile  tests.  Data  acquisition  is  done  by  fixing  the 
adhesive  markers  at  the  top  of  the  wave  profile  at  each 
x-station.  The  model  is  removed  from  the  tank  and  a 
flexible  ruler  is  used  to  measure  the  wave  profile 
distance  along  the  girth  of  the  model  from  the  calm 
waterline.  The  above  two  steps  are  repeated  three 
times.  The  model  is  inverted  and  mounted  on  a  level 
table  and  the  average  wave  height  values  are  remarked 
along  the  girth  of  the  model  from  the  calm  waterline. 
The  height  gauge  is  used  to  measure  the  wave  height  z. 

Far-field  wave  elevations.  Far-field  wave 
elevations  ^te  measured  with  either  capacitance-  or 
servo/acoustic-based  measurement  systems  or  PC  data 
acquisition.  A  longitudinal-cut  method  is  used  to 
acquire  the  far-field  data.  Data-reduction  is  completed 
by  conversion  of  longitudinal-cut  time  histories  to  a 
ship  coordinate  system  and  then  normalizing  elevations 
wilfr  model  length.  DTMB  uses  cspacitance-wire 
probes  suspended  from  an  automated  2D  traversing 
system,  a  16-bit  AD  card,  and  shore-based  PC  for  the 
far-field  wave  elevations  tests.  The  probes  and 
traversing  system  arc  cantilevered  from  the  tank 
sidewall  on  a  boom.  The  capacitance  wires,  2D 
traversing  system,  and  shore-based  PC  AD  card  are 
statically  calibrated  to  determine  their  voltage-elevation 
relationships.  Data  acquisition  is  done  through 
collection  of  2000  and  3000  discrete  samples  over  20 
and  30  seconds  at  100  Hz  for  Fr=0.28  and  0.41, 
respectively.  Data  is  filtered  at  10  Hz.  Data  was 
collected  at  two  longitudinal  positions:  y=0.097  and 
0,324.  INSEAN  uses  an  array  of  four  capacitance-wire 
probes,  moveable  slide,  12-bit  AD  card,  and  shore- 
based  PC  for  the  far-field  wave  elevation  tests.  The 
probes  and  traversing  system  are  cantilevered  from  the 
tank  sidewall  on  a  boom.  The  capacitance  wires, 
moveable  slide,  and  shore-based  PC  AD  card  are 
statically  calibrated  to  determine  their  voltage-elevation 
relationships.  Data  acquisition  is  done  through 
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collection  of  552  discrete  samples  over  12  seconds  at 
92  Hz.  Data  is  filtered  at  20  Hz,  Data  is  collected  at  a 
total  of  136  longitudinal  cuts  that  are  spaced  at 
Ay=0.0026  between  the  maximum  beam  of  the  model 
and  y==0.433.  IIHR  uses  a  Kenek  servo-type  wave 
probe  and  signal  conditioner,  a  Keyence  acoustic-type 
wave  probe  and  signal  conditioner,  2D  traversing 
system,  12-bit  AD  card,  and  shore-based  PC  for  the  far- 
field  wave  elevation  tests.  The  probes  and  traversing 
system  are  cantilevered  from  the  tank  sidewall  on  a  1.0 
m  boom.  The  servo  and  acoustic  probes,  signal 
conditioners,  2D  traversing  system,  and  shore-based  PC 
AD  card  are  statically  calibrated  to  determine  their 
voltage-elevation  relationships.  Data  acquisition  is 
done  through  collection  of  2700  discrete  samples  over 
13.3  seconds  at  202.5  Hz.  Data  is  not  filtered.  Data  is 
collected  at  a  total  of  32  longitudinal  cuts  that  are 
spaced  at  Ay=K).0l  between  the  maximum  beam  of  the 
model  and  the  sidewall  wave  dampeners  (y=0.392). 

Near-field  wave  elevations*  Ncar-ficld  wave 
elevations  are  measured  with  servo-based 

measurement  systems  and  PC  data  acquisition  at 
DTMB  and  IIHR.  A  transverse-cut  method  is  used  to 
acquire  the  data  at  the  bow,  stem,  and  wake  regions 
which  are  inaccessible  with  the  longitudinal-cut 
method.  Data-reduction  processes  are  similar  as  for  the 
carriage  speed  tests.  DTMB  employs  four  DTMB 
whisker  probes,  signal  conditioners,  2D  traversipg 
system,  16-bit  AD  card  and  carriage  PC  for  the  near- 
field  wave  elevation  tests.  The  whisker  probes,  signal 
conditioners,  and  carriage  PC  AD  card  are  statically 
calibrated  on  the  traverse  system  to  determine  its 
voltage-elevation  relationships.  Data  acquisition  is 
done  at  100  Hz  and  the  data  is  filtered  at  10  Hz.  Data  is 
collected  at  a  total  of  20  transverse  cuts  that  are  spaced 
at  Ax=0.0088  in  bow  and  stem  regions.  Data  is 
collected  with  a  continuous  traversing  method  in  the  y 
(transverse)  coordinate  with  Ay'^0.0009.  UHR  employs 
a  Kenek  servo  wave  probe,  signal  conditioner,  2D 
traversing  system,  12-bit  AD  card  and  carriage  PC  for 
the  near-field  wave  elevation  tests.  The  servo  probe, 
signal  conditioner,  and  carriage  PC  AD  card  are 
statically  calibrated  on  the  traverse  system  to  determine 
its  voltage-elevation  relationship.  Data  acquisition  is 
done  through  collection  of  4096  discrete  samples  over  9 
seconds  at  455  Hz.  Data  is  not  filtered.  Data  is 
collected  at  a  total  of  46  transverse  cuts  that  are  spaced 
at  Ax=0.05  in  bow  and  stem/wake  regions.  Data  is 
acquired  with  a  point-to-point  method  with  Ay=K).005 
between  measured  points. 

Nominal  wake.  Nominal  wake  data  (U,  V,  W, 
Cp;  xrK).935)  are  measured  with  multihole 
probe/differential  pressure  transducer-based 
measurement  systems  and  PC  data  acquisition.  DTMB 


utilizes  a  five-hole  (3.2  mm  tip),  boundary  layer  pitot 
probe,  pitot-static  probe,  five  differential  pressure 
transducers  and  signal  conditioners,  16-bit  AD  card, 
and  carriage  PC  for  the  nominal  wake  tests.  The  five- 
hole  probe  is  calibrated  before  acquisition  of  the 
nominal  wake  data.  Pitot  tube  calibration  pressure 
coefficient  matrices  arc  determined  from  the  calibration 
measurements,  performed  on  a  calibration  rig  towed  in 
calm  water  with  no  ship  model  present.  The  calibration 
is  expressed  in  coefficient  form  as:  pitch  angle  (Cp 
pitch)  versus  yaw  angles  (Cp  yaw),  and  axial  velocity 
(Cp  vel)  versus  yaw  angle.  Data  acquisition  for  these 
experiments  was  done  in  a  rectangular  coordinate  frame 
through  collection  of  500  samples  over  5  seconds  at 
100  Hz.  Data  is  collected  at  358  points  on  18 
horizontal  cuts  with  variable  spacing  in  y  and  z.  Data 
reduction  is  done  with  the  calm-water  calibration 
matrices.  INSEAN  utilizes  a  port-side,  five-hole, 
boundary  layer  pitot  probe  (3.2  mm  tip),  pitot-static 
probe,  five  differential  pressure  transducers  and  signal 
conditioners,  16-bit  AD  card,  and  carriage  PC  for  the 
nominal  wake  tests.  Three  calibrations  are  used  for  the 
nominal  wake  tests:  (1)  five-hole  pitot  probe  is 
calibrated  in  the  IIHR  1.07-m  open  throat  wind  tunnel. 

(2)  differential  pressure  transducers  and  signal 
conditioners  are  statically  calibrated  with  water  head  to 
establish  the  voltage-pressure  relationships;  and  (3) 
calibration  is  made  for  the  five-hole  pitot  probe  preset 
angles  (Op,  by  taking  initial  data  at  each  x-station 
with  the  probe  located  at  sufficient  (y,  z)  that  uniform- 
flow  conditions  prevail.  Data  acquisition  is  done 
through  collection  of  2000  samples  over  2  seconds  at 
1000  Hz.  Data  is  collected  at  a  total  of  32  horizontal 
cuts  that  are  spaced  AzK).0025.  Transverse  spacing  of 
data  is  Ay=0.0025.  Data  reduction  is  done  in  five  steps: 
(1)  AD  card  output  is  statistically  analyzed;  (2)  the 
averse  value  is  converted  to  mm  H2O  using  the 
voltage-pressure  calibration  with  linear  interpolation; 

(3)  velocity  vector  angles  (a,  0)  and  probe  calibration 
coefficients  (M,  P)  are  obtained  with  local  linear 
interpolation;  (4)  correction  for  five-hole  pitot  probe 
preset  angle  from  calibration  for  (oCp,  <|>p);  and  (5)  U,  V, 
W,  and  Cp  are  calculated.  Density  is  calculated  as 
described  for  the  resistance  test  IIHR  utilizes  the  same 
equipment  and  procedures  as  INSEAN  except:  (1) 
IIHR  uses  a  proportionately  smaller  five-hole  pitot 
probe  which  has  the  same  size  probe  tip;  (2)  IIHR 
measures  starboard-side  nominal  wake  data;  and  (3) 
IIHR  collects  1500  samples  over  12  seconds  at  125  Hz 
with  a  12-bit  AD  card, 

UNCERTAINTY  ASSESSMENT 

All  three  institutes  followed  ITTC  (Quality 
Manual  Procedures  4.9-03-01-01  Uncertainty  Analysis 
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in  EFD,  Uncertainty  Analysis  Methodology;  4,9-03-01- 
02  Uncertainty  Analysis  in  EFD,  Guideline  for  Towing 
Tank  Tests;  and  4.9-03-01-01  Uncertainty  Analysis, 
Example  for  Resistance  Test.  The  Example  for 
Resistance  Test  is  based  on  results  from  most  members 
of  the  22“*  ITTC  Resistance  Committee,  including  the 
present  results  from  INSEAN  and  IIHR.  Bias  limits  are 
estimated  with  consideration  to  significant  elemental 
error  sources  for  individual  variables,  whereas  precision 
limits  are  estimated  end-to-end  for  experimental  results 
based  on  multiple  tests  at  the  same  conditions.  Total 
uncertainties  are  estimated  with  a  root-sum-square 
(RSS)  and  normalization  with  the  range  of  the  result. 
Table  2  summarizes  the  uncertainty  assessment  results 
for  the  overlapping  tests. 

Model  geometry.  Precision  limits  are  not 
considered  in  the  analyses,  thus,  all  of  Us  is  from  the 
bias  limit.  For  DTMB  and  INSEAN,  100%  of  the  bias 
limit  is  associated  with  the  errors  in  the  model 
geometry  offsets  (x,  y,  z)  which  are  averaged  over  25  x- 
stations  with  templates.  For  IIHR,  inaccuracies  in 
loading  the  model  to  the  correct  draft  account  for  78% 
of  Us,  and  the  remaining  22%  are  associated  with  the 
errors  in  the  model  geometry  offsets  which  are 
averaged  over  31  x-stations  with  templates.  The  error 
in  the  geometry  offsets  is  small  (Bx,y.  2=0.8  mm),  but  Us 
is  relatively  high  because  the  model  is  comparatively 
small,  having  3.5-times  less  wetted  surface  area  than 
5415  and  2340A. 

Carriage  speed.  Uncertainty  in  U  at  DTMB 
is  very  small  (0,03%)  for  low  and  medium  Fr.  Bias 
errors  stemming  from  measurement  of  D  and  n  are 
major  contributors  to  the  bias  limit.  Precision  limits 
contribute  moderately  and  increase  with  increasing  Fr. 
Uncertainty  in  Uc  at  INSEAN  is  one  order  of  magnitude 
larger  than  at  DTMB  and  decreases  with  increasing  Fr, 
As  per  DTMB,  measurement  uncertainty  in  D  and  n  are 
significant  and  account  for  most  of  the  bias  limits. 
Precision  limits  are  relatively  low  and  not  Fr-dependent 
which  suggests  that  the  drive  motor  for  INSEAN 
towing  tank  no.  2  is  stable  at  all  speeds  in  the  test 
matrix.  Uncertainty  in  Ut  at  IIHR  is  somewhat  higher 
than  at  INSEAN  due  to  the  lower  range  of  speeds  for 
the  smaller  model.  Uuc  also  decreases  with  increasing 
Fr  as  per  INSEAN.  For  all  Fr,  bias  limits  dominate  (82- 
94%)  Uuc  and  decrease  with  increasing  Fr, 
Measurement  of  n  occurs  twice  in  the  data  stream  and 
accounts  for  95-100%  of  the  bias  limits.  Contributions 
from  measurement  of  D  are  0.2-3.5%  and  errors  in  the 
AD  timebase  are  negligible  as  per  DTMB  and 
INSEAN.  Puc  increases  with  increasing  Fr  indicate 
reduced  ability  of  the  drive  motor  to  maintain  constant 
Uc  for  increasing  speeds,  i,e.,  for  increasing  load  on  the 
motor. 


Resistance.  Uncertainty  in  Cy  ^  at  DTMB 
is  of  same  order  as  values  presented  by  22“^  ITTC 
Resistance  Committee,  4.9-03-01-01  Uncertainty 
Analysis,  Example  for  Resistance  Test.  Uncertainty  in 
the  loadcell  calibration  weight  standard  produces  high 
and  moderate  contributions  of  bias  limit  to  total 
uncertainty  for  Fr=0.l0  and  0.28,  respectively. 
Precision  limit  contributions  are  relatively  high  which 
is  possibly  due  to  residual  tank  motions  between  runs. 
Uncertainty  in  at  INSEAN  is  moderately 

higher  than  at  DTMB  and  decreases  with  increasing  Fr. 
Biases  in  the  measurement  of  F*  contribute  70%-10% 
of  Bcti5  for  increasing  Fr  and  are  attributed  to  the 
standardized  calibration  weights  and  scatter  in  the 
calibration  data.  Other  significant  faaors  in  Bcris  are 
Us  which  contributes  10%-65%  for  increasing  Fr  and 
Uuc  which  contributes  5%-15%  for  increasing  Fr.  The 
uncertainties  in  the  water  temperature  measurement  are 
negligible  for  all  Fr  as  per  DTMB  and  IIHR.  Precision 
limit  contributions  are  relatively  high  as  per  DTMB 
which  also  may  be  possibly  due  to  residual  tank 
motions  between  runs.  Uncertainty  in  at  IIHR 

is  roughly  similar  as  for  INSEAN  except  for  low  Fr 
where  IIHR  is  46%  lower  than  INSEAN,  The  bulk  of 
UcTi5  is  bias  related  with  Uuc  and  Us  contributing  62%- 
29%  to  Bcti5  for  increasing  Fr  and  9%-71%  to  Bens  for 
increasing  Fr,  respectively.  Effects  of  scatter  in 
loadcell  calibration  data  account  for  100%  of  Bp*. 
Precision  limit  contributions  are  relatively  low  for  all  Fr 
which  may  be  due  to  less  residual  tank  motions 
between  runs  as  compared  with  DTMB  and  INSEAN, 
Sinkage  and  trim.  Uncertainty  in  a  and  x  at 
DTMB  is  somewhat  large  for  Fr=0.10  but  decreases 
significantly  with  increasing  Fr  as  the  range  of  the 
measurements  increase,  i.e.,  the  potentiometers  operate 
further  from  their  limiting  resolution  for  higher  Fr. 
Bias  limit  contributions  for  both  variables  are  mainly 
affected  by  the  scatter  in  the  potentiometer  calibrations 
and  decrease  with  increasing  Fr.  Conversely,  precision 
limit  contributions  increase  with  increasing  Fr  which 
may  indicate  elevated  carriage  vibration  for  higher  Fr 
and  or  residual  tank  motions.  Uncertainty  in  a  and  t  at 
INSEAN  is  large  for  Fr=O.10  but  decreases 
significantly  to  values  comparable  with  DTMB  with 
increasing  Fr.  Bias  limits  are  negligible  for  both 
variables  and  all  Fr  due  to  dominance  of  precision 
limits  whose  elevated  values  are  attributed  to  residual 
tank  motions  between  carriage  runs.  Uncertainty  in  c 
and  t  at  IIHR  is  somewhat  large  for  Fr=O.10  but 
decreases  significantly  with  increasing  Fr  as  per  DTMB 
and  INSEAN.  Bias  limit  contributions  for  <5  are  high  to 
moderate  for  Fr=0.10  and  Fi?=0.28/0.41,  respectively, 
with  no  Fr-dependence.  Bias  limit  contributions  for  x 
are  high  at  Fr=0.10  and  decrease  with  increasing  Fr  as 
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per  DTMB.  Bias  limit  magnitudes  originate  from 
scatter  in  the  calibration  data.  Precision  limit 
contributions  are  mostly  high  for  both  variables  and  all 
Fr  and  caused  by  residual  tank  motions  and  heave/pitch 
oscillations  at  the  model  mount  as  the  model  seeks  its 
hydrodynamic  equilibrium.  This  ‘porpoising’  of  the 
model  is  likely  present  at  all  facilities. 

Wave  profile.  Uncertainty  in  ^  is  less  than 
5%  for  all  facilities  and  both  Fr  and  decreases  with 
increasing  Fr.  For  DTMB  and  both  Fr,  bias  limits 
account  for  64.5%  of  and  are  composed  of  error 
estimates  for  marking  the  wave  profile  at  the  exact  air- 
water  interface  (85%)  and  measuring  the  distance  from 
draftline  to  measurement  with  a  vernier  caliper  (15%). 
Precision  limits  account  for  35.5%  of  U;  and  are  based 
on  three  measurements  of  the  wave  profiles  and 
estimates  of  the  contact-line  unsteadiness.  For 
INSEAN  precision  limits  are  not  considered  in  the 
analysis.  Contributions  to  the  bias  limits  are  associated 
with  the  uncertainties  in  the  hull-gridlinc  thickness 
(40%),  optical  distortion  for  the  camera  (50%),  scanner 
resolution  (5%),  and  interpolation  error  (5%).  For  IIHR 
and  both  Fr,  bias  and  precision  limit  contributions  to  U; 
are  roughly  80%  and  20%,  respectively.  Contributions 
to  the  bias  limits  are  associated  with  uncertainties  in  the 
adhesive  marker  placement  on  the  hull  (50%), 
placement  of  the  draft  and  x-station  lines  on  the  model 
(23%),  reapplication  of  the  wave  profile  marks  on  the 
hull  after  the  test  when  the  model  is  removed  from  the 
tank  (23%),  and  height-gauge  readings  from  draftline  to 
measurement  (4%).  Precision  limit  contributions  to  U; 
are  relatively  low  and  computed  for  N=3  multiple  tests. 

Far-field  wave  elevations.  Uncertainty  in 
at  DTMB  is  better  than  4%  for  Fr=0.28  and  0.41.  A 
longitudinal  cut  at  y=0.082  is  chosen  for  detailed 
assessment  of  measurement  uncertainty.  For  both  Fr, 
bias  limits  are  main  contributor  to  U(;ff  and  are 
composed  mainly  of  scatter  in  the  calibration  data. 
Precision  limits  are  estimated  from  N=9  multiple  tests 
and  increase  with  Fr  which  may  be  due  to  elevated 
levels  of  frcc-surfacc  turbulence  with  increasing  Fr  and 
“snapshot”-like  feature  of  the  longitudinal-cut  method. 
Uncertainty  in  ^ff  at  INSEAN  is  3%  or  better  for 
Fr=0.28  and  0.41.  Longitudinal  cuts  at  y=0.082,  0.172, 
0.259,  and  0.347  are  chosen  for  detailed  analysis  of 
measurement  uncertainty.  The  majority  of  U;ff  is  bias- 
limit  related.  The  uncertainty  in  Uc  accounts  for  70% 
of  B;ff  with  a  10%-15%  contribution  from  the 
uncertainty  in  (x,  y)  probe  position  in  the  test  region. 
Uncertainty  in  the  distance  measurement  D  between  the 
wave  probes  and  the  FP  of  the  model  at  t=0  accotmts 
for  5%  of  B;ff-  Uncertainty  in  the  time  lag  between 
switch  engagement  and  data  acquisition  is  negligible. 
Precision  limits  are  estimated  with  N~10  multiple  tests. 


contribute  moderately  to  U^ff,  and  decrease  with  Fr  in 
opposition  to  those  at  DTMB.  Uncertainty  in  ^f  at 
IIHR  is  better  than  3.5%  for  Fr=0.28.  A  longitudinal 
cut  at  y=0.082  is  chosen  for  detailed  analysis  of 
measurement  uncertainty.  The  majority  of  U^ff  is  bias- 
limit  related.  The  uncertainty  in  carriage  speed 
accounts  for  79%  of  B^ff  with  6%-8%  contribution 
from  the  uncertainty  in  (x,  y)  probe  positioning  in  the 
test  region.  Uncertainty  in  the  distance  measurement  D 
between  the  wave  probes  and  the  FP  of  the  model  at  t=0 
accounts  for  6%  of  B;ff-  Uncertainty  in  the  time  lag 
between  switch  engagement  and  data  acquisition  is 
negligible.  Precision  limits  are  estimated  with  N=10 
multiple  tests  and  contribute  significantly  to  U;ff  as  per 
DTMB  and  INSEAN. 

Near-field  wave  elevations.  For  DTMB,  low- 
turbulence  LTR  and  high-turbulence  HTR  (x,  y)  regions 
are  identified  for  detailed  analysis.  In  the  HTR, 
precision  limit  contributions  to  U;nf  are  elevated 
(85.5%)  from  large  fluctuations  in  the  free  surface  and 
air  entrainment  into  the  flow.  This  produces 
comparatively  high  uncertainty  in  the  near-field 
measurements.  For  the  LTR,  contributions  of  bias  and 
precision  limits  are  more  balanced  and  the  uncertainty 
is  less  than  5%.  For  this  case,  the  scatter  in  the  whisker 
probe  calibration  governs  bias  limit  magnitude.  For 
IIHR,  LTR  (x,  y=0.05,  0.07)  and  HTR  (x,  y=1.075,  0) 
regions  are  identified  in  the  wavefield  for  detailed 
analysis.  Bias  limits  contribute  (25-50%)  to  U^nf 
which  is  mainly  (^-100%)  due  to  scatter  in  the  servo- 
probe  calibration  data.  Relatively  high  Pi;nf  is  due  to 
fiw-surface  turbulence  at  the  multiple-test  conditions. 
Note  that  P;nf  is  thrcc-timcs  greater  in  the  HTR  than 
the  LTR 

Nominal  wake.  For  DTMB,  LTR  (x,  y, 
z-0.9346,  0.04,  -0.065)  and  HTR  (x,  y,  z-0.9346, 0.02, 
-0,02)  regions  are  identified  in  the  flowfield  for  detailed 
assessment  of  the  measurement  uncertainty.  In  both 
HTR  and  LTR,  bias  limit  contributions  to  total 
uncertainties  are  dominant  for  U  and  Q  but  somewhat 
evenly  matched  with  precision  limit  contributions  for  V 
and  W,  Uncertainties  in  the  five-hole  calibration 
accounts  or  23%  of  the  bias  limit  with  the  remaining 
portion  due  to  probe  position  uncertainty  in  die 
flowfield  and  pressure  stabilization  during  a  carriage 
run.  Precision  limits  are  estimated  from  N==10  multiple 
tests  and  are  moderate  for  V,  but  15-35%  of  the  total 
uncertainty  for  U,  W,  and  Cp  which  may  be  a  result  of 
high  fr*ee-surface  turbulence/probe  vibration.  For 
INSEAN,  LTR  (x,  y,  z-0.9346,  0.06,  -0.0602)  and 
HTR  (x,  y,  z=0.9346,  -0.0025,  -0.0602)  regions  are 
identified  in  the  flowfield  for  detailed  assessment  of  the 
measurement  uncertainty.  Bias  and  precision  limit 
contributions  to  total  uncertainties  of  all  variables  are 
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evenly  distributed  in  the  LTR  and  HTR.  Bu  and  Bcp 
are  influenced  mostly  by  uncertainties  in  wind  tunnel 
coefficients,  M  and  P,  respectively.  For  crossflow 
velocities,  By  and  Bw  are  mainly  affected  by 
uncertainties  in  velocity  veaor  pitch  and  yaw  angles  (a, 
<|)).  Contributions  from  uncertainties  in  Uc  to  all 
variables  arc  less  than  3,5%,  and  uncertainties  in 
measurement  of  pressures  at  the  probe  tip  are 
negligible,  except  for  the  center  hole  where  roughly 
12%  and  2%  contributions  to  Bu  and  Bcp  are  computed. 
Precision  limits  are  estimated  from  N=10  multiple  tests 
and  are  moderate  and  high  for  U  and  V,  respectively, 
but  10-30%  for  W  and  Cp  which  may  be  caused  as  per 
DTMB.  For  IIHR,  LTR  (x,  y,  2-0.9346,  0.1,  -0.01375) 
and  HTR  (x,  y,  z=0.9346,  0,  -0,02125)  regions  are 
identified  in  the  flowfield  for  detailed  assessment  of  the 
measurement  uncertainty.  Bias  limit  contributions  to 
total  uncertainties  of  all  variables  are  dominant  in  the 
LTR  and  HTR  In  the  LTR,  By  and  Bcp  are  influenced 
mostly  by  uncertainties  in  wind  tunnel  coefficients,  M 
and  P,  respectively.  By  and  Bw  are  mainly  affected  by 
uncertainties  in  velocity  vector  pitch  and  yaw  angles  (a, 
<t)).  In  the  HTR,  Bu,  By,  Bw,  and  Bcp  are  mainly 
influenced  by  uncertainties  in  the  measurement  of  water 
head  at  the  five-hole  probe  tip  since  the  size  of  the  tip  is 
large  with  respect  to  the  shear-flow  gradients.  By  and 
Bw  are  also  still  affected  significantly  by  uncertainty  in 
the  pitch  and  yaw  angles  in  the  HTR,  Precision  limits 
are  estimated  from  N=10  multiple  tests  and  are  very 
small  in  relation  to  the  bias  limits,  which  may  be  due  to 
lack  of  free-surface  turbulence/pitot-probe  vibration. 

CFD  VALIDATION/COMPLEMENTARY  CFD 

The  conditions  and  data  locations  and  densities 
(Table  1)  were  selected  with  consideration  to  use  of 
data  for  CFD  validation.  Cruise  (Fr==0,28)  and  flank 
(Fr=0.41)  speeds  were  selected  for  detailed  validation 
with  most  extensive  tests  for  former  condition.  No 
specific  requirement  was  placed  on  experimental 
uncertainties  Ud,  but  rather  considered  an  important 
quantity  to  be  estimated  at  each  facility  with  final 
estimates  based  on  collective  results,  as  discussed 
below.  Additionally,  previous  experience  and 
complementary  CFD  was  used  for  determining  data 
densities. 

COMPARISON  OF  RESULTS 

The  focus  of  the  discussions  is  on  the 
comparison  between  institutes  of  facilities,  model 
geometry,  and  overlapping  test  results  for  evaluation  of 
facility/modcl  geometry  and  scale  effect  biases. 
Comparisons  are  made  of  measurement  systems  and 
procedures  and  data  (A,  B,  C)  [equation  (7)X  data 
differences  A-  [equations  (8),  (10),  (12)1  ^ata 


uncertainties  U(a,b.c)>  and  data-difference 
uncertainties  C/£).  [equations  (9),  (II),  (13)].  Data 
differences  for  all  variables  are  computed  after 
interpolation  of  two  data  sets  onto  standard  dependent 
variable  values  and  subtraction.  Average  data 
differences  are  expressed  as  percentages  by  normalizing 
the  average  difference  wirii  the  average  range  (all 
institutes)  of  the  variable. 

Facilities.  Facility  locations  are  different  as 
manifest  by  latitude,  climate,  and  zone.  Yearly  average 
high/low/dew  point  temperatures  are  66.6748.8744.4®, 
68.0®/51,9®/52.8®,  and  59,8®/39.7®/39.5®,  respectively,  at 
A,  B,  C.  A  and  B  have  similar  mild  yearly  climates, 
whereas  C  experiences  harsher  climate,  especially 
during  the  fall  and  winter  months.  Zones  for  A  and  B 
are  on  outskirts  of  large  cities,  whereas  C  is  in  the 
center  of  a  small  city.  Such  differences  in  ambient 
conditions  are  partially  accounted  for  by  using  values  of 
g  and  density  and  viscosity  based  on  local  values  of 
latitude  and  water  temperature,  respectively.  Water 
quality  is  also  different  at  each  facility.  A  and  C  use 
local  tap  water  and  B  uses  local  spring  water;  however, 
no  account  is  made  for  water  quality  since  as  already 
mentioned  all  institutes  base  values  of  density  and 
viscosity  on  water  temperature  only  and  fresh  water 
values  from  standard  tables. 

The  tanks  at  each  facility  have  different 
dimensions  with  the  largest  to  smallest  being  A,  B,  and 
C,  respectively,  which  affects  both  blockage  and 
residu^  tank  water  motions  between  carriage  runs.  The 
blockage  values  (ratio  of  beam/draft  product  and  tank 
cross-sectional  area)  at  A,  B,  and  C  are  0.0017,  0,0042, 
and  0,0055,  respectively,  which  are  all  nearly  at  or 
below  the  recommended  maximum  value  of  0.005.  It  is 
also  recommended  that  model  length  should  not  be 
greater  than  tank  depth  or  more  than  half-as-long  as 
tank  width.  Thus  only  A  satisfies  all  three  requirements 
and  B  and  C  each  violate  one.  Effects  of  blockage  arc 
partially  accounted  for  through  correction  to  carriage 
speed.  Tank  size  and  side  and  end  wail  wave  damping 
affect  residual  tank  water  motions  (free  and  sub 
surface)  and  required  time  intervals  between  carriage 
runs.  A  and  B  require  20  minutes  between  carriage 
runs  and  C  12  minutes  all  based  on  visual  observations. 
In  spite  of  waiting  times,  low  frequency  motions  are 
still  evident  especially  in  the  larger  tanks  (as  mentioned 
above  with  regard  to  large  precision  limits  for  sinkage 
and  trim  measurements). 

Carriage  speed  affects  all  results,  although  not 
always  included  directly  in  data-reduction  equations. 
Although  each  institute  measures  carriage  speed  with 
similar  instrumentation  (encoders  and  pulse  counters), 
they  have  different  resolutions,  A  has  the  lowest 
angular  resolution  in  their  magnetic  encoder  (520 
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pulses/rev)  but  the  lowest  uncertainty,  while  C  has  the 
highest  resolution  but  the  highest  uncertainty.  The 
uncertainty  differences  are  due  to  the  procedures  for 
transferring  pulse  count  into  the  data-acquisition  PC 
(frequency  for  A  and  two  AD  conversions  for  C)  and 
the  speed  range  for  the  uncertainty  assessment.  Since  A 
and  B  operate  at  nearly  twice  the  carriage  speed  for  a 
given  Fr,  and  speed  range  is  the  normalizing  factor  for 
determination  of  TJuc,  A  and  B  have  less  uncertainty  in 
measured  Uc.  Uuc  can  be  further  reduced  through 
implementation  of  a  closed-loop  feedback  system  that 
constantly  assesses  and  updates  with  respect  to  a 
desired  value.  Currently,  B  is  adopting  this  capability, 
however,  none  of  the  facilities  had  this  capability 
during  the  overlapping  tests.  Carriage  ride  affects 
results  through  carriage  vibration;  however,  such 
effects  were  not  considered. 

Model  geometry.  Model  geometry  offsets, 
bow  details  (leading-edge  radius),  turbulence 
stimuiation,  surface  roughness,  and  installation  also 
surely  affect  all  results,  although,  here  again,  not 
always  directly  through  data-reduction  equations.  For 
each  institute,  Us  is  estimated  to  be  0.5%  which  is  near 
the  level  of  Us  reported  by  the  participating  members  of 
the  22^  ITTC  uuceilaiiily  assessuieiil  example  for 
resistance  test.  The  method  for  determination  of  the 
tmeertainties  in  the  offsets  is  crude  as  templates  are 
used  at  a  limited  number  of  stations  and  error 
estimation  is  tedious,  involving  moderate  and  low 
accuracy  at  low-  and  high-points  on  the  hull  surface, 
respectively.  Accounting  for  twisting  or  sagging  of  a 
model  in  the  estimation  of  the  offset  errors  is  a  very 
difficult,  if  not  impossible  task  with  ten^lates.  Also 
complicating  the  issue  is  changes  to  the  model  offsets 
over  time  with  changes  in  ambient  temperature  and 
humidity  or  by  long-term  water-immersion.  Leading- 
edge  radius  on  each  model  is  3.2  mm  for  A  and  B  and 
1,6  mm  for  C.  Turbulence  stimulation  is  different  at 
each  facility  in  terms  of  tripping  location,  however, 
results  in  suggest  that  the  turbulence 

stimulation  was  effective  for  all  three  models.  Model 
surfaces  were  finished  using  usual  procedures  at  each 
facility  and  assumed  hydraulically  smooth.  All  models 
were  installed  according  to  draft  line,  which  is 
presumed  the  best  method  for  CFD  validation  purposes. 
However,  it  was  noted  by  22“*  ITTC  uncertainty 
assessment  example  for  resistance  test  that  installation 
according  to  ballast  weight  reduces  uncertainty  in 
surface  area  by  partially  accounting  for  inaccuracy  of 
design  offsets.  Tow  points  are  also  different  for  each 
model,  but  have  not  yet  been  compared. 

Resistance.  For  resistance  tests,  measurement 
systems,  procedures,  and  data  uncertainty  estimates 
(Table  2)  are  similar  for  all  three  institutes.  Figure  2 


displays  the  results.  Figure  2a  compares  the  resistance 
data  and  Figures  2b,  c,  d  compare  the  data  difference 
and  data-difference  uncertainties  between  institutes  AB, 
AC,  and  BC,  respectively.  Trends  in  Figure  2a  for  all 
three  institutes  are  typical  for  high-speed  combatant, 
i.e,,  limited  humps  and  hollows  for  low  and  medium  Fr 
and  sharply  increasing  resistance  for  high  Fr.  For  low 
and  medium  Fr  results  for  A,  B,  and  C  are  very 
consistent  with  C  usually  between  A  and  B,  whereas  for 
high  Fr  C  is  consistently  lower  than  A  and  B. 

0.006  r— - 1 


O.OOOI  I  I  I  I  I  t.  i.t  I  I  I  t  I'l  »  i  i  '  i  M  i 

0.0  0.1  0.2  0.3  0.4  0.5 

(a)  Fr 


Fig.  2:  Resistance  results  and  data  differences. 


For  AB  (Figure  2b),  data  differences  are 
oscillatory  with  Fr  and  intermittently  greater/less  than 
or  within  data-difference  uncertainty,  which  suggests 
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unaccounted  for  bias  and  precision  limits.  Average 
data  difference  is  2.2%,  whereas  average  data- 
difference  uncertainty  is  only  1.5%. 


Fig.  3:  Sinkage  results  and  data  differences. 


For  AC  (Figure  2c)  and  low  and  medium  Fr, 
trends  are  similar  (i.e.,  data  differences  are  oscillatory 
and  intermittently  greater/less  than  or  within  data* 
difference  uncertainty),  whereas  for  high  Fr  the  trend  is 
evident  that  data  difference  increases  nearly  linearly 
and  is  much  greater  than  data-difference  uncertainty. 
Average  data  difference  and  data-difference  uncertainty 
are  3.4%  and  1.1%,  respectively.  Former  is 


considerably  larger  than  for  AB  and  latter  is  somewhat 
smaller.  Clearly  the  differences  for  AC  compared  to 
AB  for  larger  Fr  are  due  to  scale  effects.  Trends  for  BC 
(Figure  2d)  are  very  similar  as  for  AC  with  average  data 
differences  and  data-difference  uncertainties  of  3.9% 
and  1.6%,  respectively. 


Fig.  4:  Trim  results  and  data  differences. 


Results  are  important  in  showing  that  for  same 
size  model  (5.72  m)  a  better  estimate  for  uncertainty  in 
resistance  test  is  2.2%,  which  is  considerably  larger 
than  individual  facility  estimates,  especially  for 
medium  and  high  Fr.  Results  also  show  that  scale 
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effects  for  3  m  model  are  significant  for  Fr>0.26,  which 
is  likely  due  to  differences  in  wave  breaking  as  will  be 
discussed  later  with  regard  to  wave  elevation 
measurements.  For  resistance,  average  facility/model 
geometry  and  scale  effect  (Fr>0.26)  biases  are 
estimated  as  0.5%  and  5.8%,  respectively,  as 
summari2x:d  in  Tabic  3.  Note  that  averages  arc  based 
on  all  three  facilities  and  Fr  ranges  as  given  in  Table  3, 
and  that  facility  and  model  geometry  biases  are 
combined  as  they  cannot  be  separated  without  use  of  a 
standard  model. 

Table  3:  Summary  of  facilily/model  geometry  and 
scale  effect  biases. 


Result 

Fac./mcMic;l  geomeuy  (Uf/mg) 

AB  AC  BC 

Scale  (Use)  | 

AC  BC  1 

Cr 

0.7% 

0.9%t 

0%t 

4.9%7 

6.7%? 

0 

0% 

0% 

0% 

0% 

0% 

X 

0% 

o%t 

0%t 

11.6%? 

12.7%? 

1,0% 

0% 

0% 

0% 

0% 

Co41 

2.6% 

1.9% 

0% 

0% 

0% 

2.9% 

1.1% 

0% 

0% 

0% 

U 

0% 

0% 

0% 

0% 

0% 

v 

0.2% 

0% 

2.3% 

0% 

0% 

w 

13.0% 

0% 

11.7% 

0% 

0% 

t:  Fr<0.26;  %:  Fr<0.33;  ?:  Fr>0.26;  ? :  Fr>0 J3 


Sinkage  and  trim.  For  sinkage  and  trim  tests, 
measurement  systems  and  procedures  are  similar  for  all 
three  institutes.  However,  data  uncertainties  are  fairly 
large  and  show  some  differences  (Table  2). 
Uncertainties  for  B  at  low  Fr  are  very  large.  Figures  3 
and  4  display  the  results.  Figures  3a  and  4a  compare 
the  sinkage  and  trim  data  and  Figures  3b,  c,  d  and  4b,  c, 
d  compares  the  data  difference  and  data^lifference 
imcertainties  between  institutes  AB,  AC,  and  BC, 
respectively.  Trends  in  Figures  3a  and  4a  for  all  three 
institutes  are  also  typical  for  high-speed  combatant,  i.e., 
increasing  sinkage  and  bow  down  than  up  trim  for 
increasing  Fr.  In  general,  results  between  institutes  are 
consistent,  however,  for  trim  and  high  Fr,  C  is 
consistently  lower  than  A  and  B. 

For  AB  (Figures  3b  and  4b),  data  differences 
are  oscillatory  and  mostly  within  the  data-diffeience 
uncertainties.  Average  data  differences  are  4%  and 
2.5%,  whereas  average  data-difference  uncertainties  are 
14.1%  and  18.3%,  respectively,  for  sinkage  and  trim. 
In  this  case,  data  differences  are  fairly  small  but  with 
large  data-difference  uncertainties.  For  AC  (Figures  3c 
and  4c),  trend  for  sinkage  is  similar  as  AB  although 
percentage  values  are  lower,  whereas  trend  for  trim  is 
different  for  high  Fr  wherein  as  with  resistance  test  data 
difference  increases  nearly  linearly  and  is  much  greater 
than  data-difference  uncertainty.  Average  data 
difference  and  data-difference  uncertainty  are  5.5%  and 


7.8%,  respectively.  Former  is  larger  than  for  AB  and 
latter  is  smaller.  Clearly  for  trim  test,  as  with  resistance 
test,  the  differences  for  AC  compared  to  AB  for  larger 
Fr  are  due  to  scale  effects.  Trends  for  BC  (Figures  3d 
and  4d)  are  very  similar  as  for  AC  with  average  data 
differences  and  data-difference  uncertainties  of  2.6% 
and  16.9%  and  5.5%  and  13.5%,  respectively,  for 
sinkage  and  trim. 


Fig.  5:  Wave  profile  results  and  data  differences 
(Fir=0.28). 


Results  are  in^ortant  in  showing  that  for  same 
size  model  (5.72  m)  data  differences  are  fairly  small 
(i.e.,  similar  level  as  resistance  test)  and  for  low  Fr 
considerably  less  than  data-difference  uncertainties. 
All  institutes  (especiaUy  B)  need  to  reduce  uncertainties 
for  low  Fr.  Results  also  show  that  scale  effects  for  3  m 
model  are  significant  for  trim  and  Ft>0.33. 
Facility/model  geometry  biases  are  not  evident  for  both 
sinkage  and  triixL  Scale  effect  biases  are  not  evident  for 
sinkage  and  estimated  as  12.2%  for  trim  (Table  3). 
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Wave  profile.  For  wave  profile  tests, 
measurement  systems  and  test  procedures  are  very 
different  between  institutes.  However,  data 
uncertainties  have  similar  values  (Table  2).  Figures  5 
and  6  display  the  results.  Figures  5a  and  6a  compare 
the  wave  profile  data  for  Fr  =  0.28  and  0.41  and  Figures 
5b,  c,  d  and  6b,  c,  d  compare  the  data  difference  and 
data-difference  uncertainties  between  institutes  AB, 
AC,  and  BC,  respectively.  Trends  in  Figures  5a  and  6a 
for  all  three  institutes  are  also  typical  for  high-speed 
combatant  for  medium  and  high  Fr,  i,e.,  display  bow, 
shoulder  and  stem  waves  and  increasing  transverse 
wavelength  with  increasing  Fr. 


Oi»  0^  0|0  a76  1.00 


Fig.  6:  Wave  profile  results  and  data  differences 
(Fr=0.41). 

Results  between  institutes  are  fairly  consistent, 
although  A  seems  to  show  consistently  smaller  values 
especially  for  shoulder  and  stem  waves. 

For  AB  (Figures  5b  and  6b),  data  differences 
arc  oscillatory  and  mostly  less  than  data-diffcrcncc 


uncertainties.  Average  data  differences  are  6.4%  and 
5.7%,  whereas  average  data-difference  uncertainties  are 
5.4%  and  3.1%,  respectively  for  Fr=0.28  and  0.41. 
Results  for  AC  (Figures  5c  and  6c)  and  BC  (Figures  5d 
and  6d)  are  fairly  similar,  however,  m  these  cases  data 
differences  are  mostly  within  data-difference 
uncertainties. 


Fig.  7:  Far-field  wave  elevation  results  at  Fr=0.28:  (a) 
DTMB  5415;  (b)  INSEAN  2340A;  (c)  UHR 
5512. 


Fig.  8:  Far-field  wave  elevation  differences  at 
Fi«0.28:  (a)  DTMB-INSEAN;  (b)  DTMB- 
IIHR;(c)  INSEAN-UHR. 


Results  are  important  in  showing  that  a  better 
estimate  for  uncertainty  in  wave  profile  test  is  about  5- 
6%,  which  is  larger  fimi  individual  facility  estimates, 
especially  for  high  Fr.  Results  also  show  that  scale 
effects  for  the  3  m  model  are  insignificant  Average 
facility/model  geometry  and  scale  effect  biases  are 
estimated  as  0.9%  and  not  evident,  respectively  (Table 
3). 
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Wave  elevations.  For  wave  elevation  tests, 
measurement  systems  and  test  procedures  are  different 
between  institutes.  However,  data  uncertainties  have 
similar  values  (Table  2).  Figures  7-10  display  the 
results  for  the  far  field  tests  for  Fr=0.28.  Comparisons 
for  the  near  field  tests  are  not  included.  Figure  7  and  8 
compare  the  wave  pattern  data  and  data  differences, 
respectively.  Figures  9  and  10  compare  the  data  and 
data  differences  for  two  cuts  y=0.324  and  0.082, 
respectively.  Figures  8a,  b,  c  and  9b,  c,  d  are  for  AB, 
AC,  and  BC,  respectively.  Figure  10b  is  for  BC. 
Trends  in  wave  patterns  for  all  three  institutes  are 
typical  for  high-speed  combatant,  i.e.,  show  diverging 
and  transverse  wave  systems  originating  from  lx)w, 
shoulder,  and  stem.  Overall  patterns  for  all  three 
institutes  are  very  similar,  although  resolution  for  A 
appears  less  than  that  for  B  and  C.  Figure  9a  shows 
fairly  large  differences  between  A,  B,  and  C  at  this 
distance  from  the  hull,  whereas  Figure  10a  shows  small 
differences  between  B  and  C  close  to  the  hull. 


Fig,  9:  Far-field  wave  cut  data  and  differences  at 
y*0,324,  Fr=0.28:  (a)  interpolated  wave  cuts; 
(b)  DTMB-INSEAN;  (c)  DTMB-IIHR;  (d) 
INSEAN-UHR. 


Detailed  comparisons  at  y=0.324  show  that  data 
differences  are  oscillatory  and  average  data  differences 
of  about  4-5%  and  data-difference  uncertainties  of 
about  4%  for  AB,  AC,  and  BC.  Here  again,  largest 
differences  are  at  crests  and  troughs.  Trends  are  similar 
at  y=0.082,  except  average  data  difference  is  only  0.6% 
and  data-diffcrcncc  uncertainty  is  2.8%.  In  this  case 
uncertainty  estimates  include  dependency  on  x. 


(bf^' 
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Fig.  10:  Far-field  wave  cut  data  and  differences  at 
}fO.082,  Fr=0.28:  (a)  data;  (b)  INSEAN-DHR. 


Data  differences  (Figure  8)  are  largest  for  the 
diverging  waves  at  crests  and  troughs.  For  AB,  AC, 
and  BC  average  data  differences  are  6.5%,  5.5%,  and 
2.5%,  respectively,  whereas  average  data-difference 
imcertainties  for  AB,  AC,  and  BC  are  about  4%. 


OuOB  an  Mt  0y«  ua  ^  O-Oi  OM  M  AM 

Fig,  11:  Nominal  wake  results  at  Fi?=0.28. 

Results  are  important  in  showing  that  wave 
elevation  differences  are  fiiirly  small  and  close  to  the 
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uncertainty  estimates.  Also,  scale  effects  are  not 
evident.  However,  visual  observation  of  wave  patterns 
especially  for  higher  Fr  shows  differences  in  wave 
breaking,  i.e.,  wave  breaking  is  considerably  reduced 
for  the  3  m  model  in  comparison  to  the  5,7  m  models 
presumably  due  to  differences  in  Weber  number  and 
also  some  differences  between  wave  breaking  patterns 
for  larger  models  presumably  due  to  water  quality 
differences  between  facilities  A  and  B.  Average 
facility/model  geometry  and  scale  effect  biases  are 
estimated  as  1.3%  and  not  evident,  respectively  (Table 

3). 
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Fig*  12:  Nominal  wake  data  differences  at  Fr==0.28. 

Nominal  wake.  For  the  nominal  wake  tests, 
measurement  systems  and  test  procedures  are  similar 
between  facilities.  However,  data  uncertainties  are 
fairly  large  and  show  some  differences  (Table  2). 
Figures  11  and  12  display  the  results  for  the  mean 
velocity  components  (U,  V,  W)  data  and  data 
differences,  respectively,  for  Fr=0,28.  Trends  for 
nominal  wake  for  all  three  institutes  are  typical  for 
hig^-speed  combatant,  i.e.,  relatively  thin  boimdary 
layer  near  keel  and  free  surface  and  thick  boundary 
layer  near  mid  girth  due  to  effects  of  sonar  dome 
vortex.  V  and  W  show  influences  of  sonar  some  vortex 
and  upward/inward  stem  flow.  Flow  patterns  for  all 
three  institutes  are  similar,  although  differences  in 


resolution  are  apparent.  For  C,  scale  effects  are  not 
obvious. 

For  AB,  data  differences  are  less  than  data 
difference  uncertainties  for  U,  whereas  they  are  larger 
for  V  and  W.  However,  data-difference  uncertainties 
(and  data  differences,  except  U)  are  fairly  large. 
Interestingly,  results  for  AC  and  BC  arc  similar,  i.c., 
scale  effects  are  mostly  lost  in  noise,  although  pattern 
for  data  differences  for  both  AC  and  BC  are  nearly 
same,  which  may  be  an  indication  of  scale  effects. 

Results  are  important  in  showing  that  the  data 
differences  and  data  tmeertainty  are  reasonable  close 
albeit  with  fairly  large  values.  All  institutes  need  to 
reduce  uncertainties.  Average  facility/model  geometry 
and  scale  effect  biases  are  estimated  for  (U,  V,  W)  as 
(0%,  0.8%,  8.2%)  and  0%,  respectively  (Table  3). 
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Fig.  13:  DTMB  5415  axial  velocity  (U)  contours  and 
crossflow  vectors  for  the  w/propeller 
condition:  Fr=0.28,  x=0.9603, 436  rpm. 


Fig.  14:  INSEAN  2340A  flow  mapping  at  Fr=0.28. 


mCHUGHTS  OF  OVERALL  TEST  PROGRAM 
Although  the  emphasis  herein  has  been  on  the 
overhqyping  tests,  the  collaborative  effort  also  consists 
of  focus  studies  at  each  institute.  These  studies  were 
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designed  to  address  a  variety  of  important  physics  and 
expand  the  surface-ship  database  with  quality  datasets 
and  uncertainty  assessment.  At  A,  model  5415  was 
used  in  a  series  of  propeller-hull  interaction  tests. 
Shafts  and  struts  were  added  to  the  model  and  data  was 
obtained  with  and  without  the  propellers  operating. 
The  measurements  include  free  surface  topographies  of 
the  transom  wave  field,  longitudinal  wave  cuts 
(Ratcliffe,  2000),  and  velocity  measurements  in  the 
nominal  wake  plane  and  both  upstream  and 
downstream  of  the  operating  propulsors.  The  velocity 
measurements  were  obtained  with  three-component 
laser-doppler  velocimetry  (LDV).  Fig.  13  is  a  sample 
of  results  for  the  latter.  At  B,  model  2340A  was  used 
for  a  comprehensive  flow  mapping  of  the  boundary 
layer  and  wake  flow  at  eleven  cross  planes  with  multi- 
hole  probes  and  pressure  transducers.  The  axial 

velocity  results  are  plotted  in  Fig.  14.  Results  from  this 
study  as  well  as  the  overlapping  tests  and  the  near  fleld 
bow  and  stem  wwo  clcv'ations  ftem  A  w'ill  be  used  at 
the  Gothenburg  2000  workshop  on  CFD  in  ship 
hydrodynamics.  At  C,  model  5512  is  currently  being 
u^  for  unsteady-flow  testing  in  regular  head  waves, 
Meastirements  include  unsteady  forces  and  moment 
with  a  load  cell,  fer-and  near-field  wave  ele\^tions  (Fig. 
15)  with  servo  and  acoustic  probes,  and  mean  and 
tuAulent  flow  field  with  a  towed  particle-image 
velocimetry  (PIV)  system.  The  latter  measurements  are 
ongoing  and  will  be  archived  with  the  rest  of  the 
unsteady  data  at  httD://www.iihr.uiowa.edu/"^owtank. 
Figures  11  and  12  also  include  comparisons  of  nominal 
w^e  data  for  5512  and  pitot  and  PIV  measurement 
systems.  Data  differences  are  similar  to  data-difference 
uncertainties. 

CONCLUSIONS 

Results  are  presented  fiom  overlapping  towing 
laidL  tests  between  lluee  instituies  for  resistance, 
sinkage  and  trim,  wave  profiles  and  elevations,  and 
nominal  wake  using  the  same  model  geometry  and 
conditions,  including  rigorous  plication  of  standard 
uncertainty  assessment  procedures  as  per  the  1999 
ITTC  Quality  Manual.  Two  of  the  institutes  used  5,7  m 
models  whereas  the  third  institute  used  a  smaller  3  m 
model.  Conq)arison  variables  were  defined  for  data- 
rcduction  equations  and  data  differences  and  data- 
difference  uncertainties.  Detailed  descriptions  were 
provided  of  facilities,  measurement  systems,  data- 
acquisition  and  reduction  procedures,  and  uncertainty 
assessment.  Results  were  discussed  with  regard  to 
levels  and  causes  of  data  differences  and  data- 
difference  uncertainties  and  to  estimate  facility/model 
geometry  and  scale  effect  biases. 


For  same  size  5.7  m  models,  data  differences 
were  in  general  oscillatory,  and  in  many  cases,  larger  in 
magnitude  than  data-difference  uncertainties,  which 
indicates  unaccounted  for  bias  and  precision  limits  and 
that  current  individual  facility  uncertainty  estimates  are 
often  too  optimistic.  Scale  effects  for  the  3  m  model 
arc  only  evident  for  resistance  and  trim  tests  for 
Fr>0.26  and  Fr>0.33,  respectively.  Facility/model 
geometry  and  scale  effect  bias  are  estimated  based  on 
comparisons,  as  summarized  in  Table  3.  Uncertainty 
estimates  including  such  biases  may  provide  better 
estimates,  especially  for  use  in  CFD  validation,  which 
is  the  recommendation  of  the  present  study  along  with 
efforts  towards  imprdvement  of  individual  institute 
uncertainty  estimates.  Use  of  standard  models  and 
current  ITTC  efforts  in  providing  standard  quality 
manual  procedures  for  towing  tank  tests  and  uncertainty 
estimates  will  also  be  helpful  in  this  regard. 


Fig.l5:IIHR  5512  unsteady  wave  field  at  four 
instances  in  the  encoimter  period:  Fr*0.28, 
Ak«0.025,X-4.572m. 
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Table  1:  Summary  of  overlapping  test  conditions  for  DTMB,  INSEAN,  and  IIHIL 


EXPERIMENT 

DTMB  (A) 

INSEAN  (B)  1 

MODEL  GEOMETRY 

Templates 

Templates 

CARRIAGE  SPEED 

Magnetic  encoder 

Optical  encoder 

RESISTANCE 

Fr 

Re 

T«,(X) 

p(kg/m^) 

V  (mVs) 

o(N/m) 

Datadrasity 

Model  installation 

AFP,  AAP 

Loadcell 

0.05-0.44 

0.21-1.92e07 

17.8 

998.6960 

l.0638e-06 

0.0738 

AFr=0.013 

fiee 

Loadcell 

0.05-0.45 

0.21-1 .92e07 

22.1 

997.7200 

0.9547e-06 

0.0732 

AFr-0.01 

fiee 

SINKAGEn:RIM 

Fr 

Re 

p(k^mO 
V  (nr/s) 

<y(N/m) 

Data  density 
Model  inslallatiQn 
AFP,AAP 


WAVE  PROFILE 
Fr 
Re 

T,»C*C) 

p(k^m^) 

V  (nr/s) 
c(N/m) 

Data  density 
Model  installation 
AFP.AAP 


NEAR-FIELD  WAVES 
Fr 
Re 

T^(rc) 

p(kWmO 

V  (iT/s) 

C(^) 

Data  density 
Model  installation 
AFP,AAP 


FAR-FIELD  WAVES 
Fr 
Re 

p(k^O 
V  (nr/s) 
o(N/m) 

Data  density 
Model  installation 
AFP,  AAP 


NOMINAL  WAKE 
Fr 
Re 

T«.ra 

pOc^mO 

v(ir/8) 

o(NAn) 

Data  density 
Model  insts^tion 
AFP, AAP 


linear  pots 
0.05-0.44 
0.21-1. 92e07 
17.8 

998.6960 

1.06a8e<06 

0.0738 

AFr-0.013 

free 


Wax  pencil 
028,0.41 
1.19,  l.75e07 

20.6 

998.1560 
0.9907e-06 
0.0735 
23pts,  25pts 
fixed 

(-0.0027L,  -O.OOOSeL), 
(-0.00054L,-0.0083L) 


Whisker  probe 
0.28,0.41 
1.19, 1.75e07 
20.6 

998.1560 

0.9907e-06 

0.0735 

20  cuts,  Ax«4).09L,  Ay»0.0009L 
fixed 

(4).0027L,  -0.00086L), 
(^.00054L,-0.0083L 


Capacitance  probes 
028,0.41 
1,19, 1.75e07 

20.6 

998.1560 

0.9907e-06 

0.0735 

2  cuts  at  y*O.097L  and  0.324L 
fixed 

(-0,0027L,-0.00086L), 

(-0.00054L,-0.0083L) 


5^k  probe 
0.28, 0.41 
1.19, 1.75e07 

20.6 

998.1560 

0.9907e4}6 

0.0735 

18  cuts,  358  points,  variable 
fixed 

(-0.0027L,-0.00086L), 

(-0.00054L,-0.0083L) 


Rotative  pots 
0.05-0.45 
0,2l-l.92e07 
22.1 

997.7200 

0.9547e4)6 

0.0732 

AFr-O.Ol 

fiee 


Photography 
028,0.41 
1.19, 1.75e07 
22.1 

997.7200 
0.9547e4)6 
0.0732 
41  pts 
fixed 

(-0.0027L,  -0.00086L), 
(-0.00054L,-0.0083L) 


Capacitance  probes 
0.28, 0.41 
1.19, 1.75e07 
13.3 

999.4000 

1.1923e-06 

0.0743 

136  cuts,  Ax^.0l6,  Ay*0.003 
fixed 

(-0.0027L,  .0.00086L), 
(.0.00054L,-0.0083L) 


S-hok  probe 

0.28 

I. 19e07 

II. 0 

999.6800 

1.2692e-06 

0.0744 

32  cuts,  Ay*Az*K).0025 
fixed 

(-0.0027L,  -0.00086L), 
(-O.OOOS4L,-O.0083L) 


Optical  encoder 


Loadcell 

0.05-0.45 

0.93-8.41e06 

25.1 

997.0720 

0.8925e-06 

0.0727 

AFt*0.01 

fiee 


Linear  pots 
0.05-0.45 
0.86-7.72e4)6 

21.5 

997.9400 

0.9708e-06 

0.0733 

AFiH).0I 

free 


Adhesive  marker 
028,0.41 
4.47, 6.54e06 

18.5 

998.5700 
l.0448e4)6 
0.0733 
40  pts 
fix^ 

(-0.003  IL,  .0.00079L), 
(-0.0015L.  -0.0079L) 


Servo  probe 

0.28 

3.81e06 

12.5 

999.5000 

1.224e06 

0.0745 

46  cuts,  Ax*0.05,  Ay-0.005 
fixed 

(-0.003  IL,  .0.00079L) 


Servo/acoustic  probes 
0.28 
4.48e06 

18.6 

998.5520 

1.0421e-06 

0.0737 

32  cuts,  Ax-0.001,  AjH).0l 
fixed 

(4).0031L,  -0.00079L) 


5-hok  probe 

0.28,0.41 

3.83,5.61e^6 

12.7 

998.7140 

1.2173e4)6 

0.0744 

27  cuts,  Ay-Az-0.0025 
fixed 

(-0.0031L,  -0.00079L), 
(-0.0015L,-0.0079L) 
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Table  2:  Summary  of  uncertainties  for  DTMB,  fl^SEAN,  and  DHR  overlapping  tests. 


RESULT 

Ft 

B 

DTMB  (A) 

P 

U 

INSEAN  (B) 

B  P 

U 

B 

IIHR(C) 

P 

U 

S 

AllFr 

100% 

0% 

1.2% 

100% 

0% 

0.5% 

100% 

0% 

mm 

0.10 

78.0% 

22.0% 

0.03% 

99.1% 

0.9% 

94.2% 

5.8% 

B9 

0.28 

68.2% 

31.8% 

0.03% 

93.7% 

6.3% 

85,9% 

14.1% 

■■ 

0.41 

NA 

NA 

NA 

99.5% 

0.5% 

82.3% 

17.7% 

CtiVCh 

0.10 

76.3% 

23.7% 

1.49% 

69.4% 

30.6% 

2.68% 

87.6% 

12.4% 

i^% 

Ctis^Cr 

45.5% 

54.5% 

0.33% 

80.0% 

0.64% 

89.2% 

10.8% 

0.63% 

Ctii/Cr 

NA 

NA 

NA 

66.3% 

33.7% 

0.61% 

80.5% 

19.5% 

0.60% 

■■ 

75.6% 

24.4% 

12.2% 

0% 

42% 

82.2% 

17.8% 

8.72% 

■I 

68.4% 

32.6% 

S.6% 

0% 

100% 

4.71% 

30.4% 

69.6% 

1.40% 

IkH 

0.41 

56.3% 

44.7% 

2.5% 

0% 

2.93% 

42.8% 

57.2% 

0.61% 

64.5% 

35.5% 

14.4% 

0% 

32% 

50.8% 

49.2% 

10J2% 

t 

0.28 

54.7% 

46.3% 

2.8% 

0% 

4.70% 

36.1% 

63.9% 

1.83% 

T 

0.41 

38.1% 

61.^^ 

1.5% 

0% 

100% 

0.87% 

4.1% 

95.9% 

1.76% 

0.28 

64.5% 

35.5% 

3.52% 

0% 

4.18% 

83.7% 

16.3% 

3.43% 

? 

0.41 

64,5% 

35.5% 

1.84% 

100% 

0% 

2.59% 

81.6% 

18.4% 

2.00% 

Cw"™ 

0.28 

■QjH 

NA 

NA 

NA 

25.2% 

74.8% 

338% 

Cw™ 

0.28 

MH 

NA 

NA 

NA 

52.6% 

47.4% 

4.75% 

0.41 

wss 

NA 

NA 

NA 

NA 

NA 

NA 

64.9% 

35.1% 

2.40% 

59.0% 

41.0% 

3.42% 

■■ 

78.5% 

21.5% 

3.00% 

NA 

NA 

NA 

mm 

0.28 

74.5% 

26.5% 

12J% 

60.4% 

39.6% 

0J8% 

99.2% 

0.8% 

3.11% 
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P 

43.5% 

56.5% 

6.5% 

15.9% 

84.1% 

2.71% 

93.3% 

6.7% 

3.76% 

Hfl 

I 

44.6% 

35.4% 

3.7% 

87.9% 

12.1% 

0.86% 

99.2% 

0.8% 

448% 

[kH 

T 

84.5% 

15.5% 

2.4% 

81.8% 

18.2% 

9.03% 

100% 

0% 

3631% 

0 

65.4% 

34.4% 

1.6% 

47.8% 

52.2% 

0.42% 

99.8% 

mi 

54J% 

46.7% 

2.9% 

21.2% 

78.8% 

ijrr% 

99.7% 

mm 

65.3% 

34.7% 

6.5% 

79.1% 

20.9% 

0.95% 

99.9% 

88.7% 

21.3% 

1.2% 

msm 

29.8% 

9.11% 

100% 

WWW 

u 

0.28 

42.1% 

57.9% 

235% 

ImM 

P 

72.4% 

27.6% 

7.73% 

lU 

I 

62.4% 

37.6% 

437% 

V 

20.8% 

79.2% 

4.72% 

34.9% 

65.1% 

431% 

In 

37.3% 

62.7% 

439% 

tnr 

13.0% 

87.0% 

4.09% 

aw 

30.1% 

69.9% 

530% 
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out  of  the  same  magnetoelastic  material,  which  has  reaction  times 
in  the  order  of  a  few  micro-seconds.  The  response  time  of  such  a 
design  will  be  primarily  limited  by  the  inductance  of  the  magnetic 
coil. 
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Discussion  of  the  Problem 

The  possibility  of  oscillatory  behavior  of  the  value  of  a  com¬ 
puted  variable  as  grid  size  is  refined  in  a  simulation  raises  ques¬ 
tions  in  the  interpretation  of  grid  convergence  studies  that  have,  to 
the  authors’  knowledge,  never  been  addressed.  Such  oscillatory 
behavior  has  been  observed  by  the  authors,  for  example,  in  vari¬ 
ables  such  as  wave  profiles  along  the  hull  and  wave  elevations  in 
the  stem  flow  in  simulations  of  flows  about  ships  with  complex 
geometries.  Roache  [1],  in  his  comprehensive  presentation  and 
critique  of  work  up  to  that  time  in  the  area  of  verification  and 
validation  of  simulations,  points  out  that  “behavior  far  away  from 
asymptotic  convergence  can  be  non-monotone”  and  “the  addi¬ 
tional  assumption  of  monotone  tmncation  error  convergence  in 
the  mesh  spacing . . .  may  not  be  valid  for  coarse  grids,  or  possi¬ 
bly  other  conditions.” 
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Typically  (see  Stem,  et  al.  [2],  for  instance),  the  behavior  of  a 
variable  is  categorized  as  monotonically  convergent,  oscillatory, 
or  divergent  based  on  its  behavior  as  the  grid  used  in  a  simulation 
is  refined.  Consider  the  values  of  a  computed  variable  y  as  a 
simulation  is  mn  using  a  coarse  grid  (y^),  a  medium  grid  (yM)» 
and  a  fine  grid  (y/r).  The  ratio 

^_yM-yF  _^yM-F 

yc-yu  ^yc-M 

has  been  used  to  categorize  the  behavior  of  y  as  grid  size  is  de¬ 
creased  as:  (1)  monotonically  convergent  when  0</?<l;  (2)  os¬ 
cillatory  when  R<0;  and  (3)  divergent  when  R>  1.  If  the  behav¬ 
ior  really  is  monotonically  convergent,  then  (1)  holds.  If  the 
behavior  really  is  divergent,  then  (3)  holds.  The  problem  that  has 
not  been  previously  recognized  and  discussed  is  the  ambiguity 
that  arises  when  the  behavior  really  is  oscillatory. 

Consider  the  results  from  a  simulation  in  which  the  computed 
value  of  variable  y  is  tmly  an  oscillatory  function  of  grid  size  Ajc, 
as  shown  in  Fig.  1.  When  a  grid  size  is  chosen  in  a  simulation,  its 
value  is  of  course  arbitrary  relative  to  the  unknown  periods  of  any 
oscillations  of  the  computed  variables,  so  each  of  the  following 
cases  in  this  example  must  be  considered  equally  likely.  Three 
cases  are  investigated,  with  a  different  initial  grid  size  in  each.  In 
each  case,  three  simulations  are  run  with  grid  doubling  used  twice, 
resulting  in  coarse  (C),  medium  (M),  and  fine  (F)  grid  simulation 
values  of  the  variable  y. 

The  computed  values  of  y  are  shown  in  Table  1  and  plotted  in 
Fig.  1.  For  Case  1,  ^yc-M  Is  —1.0  and  Ayjv/^/r  is  +0.3,  a  situa¬ 
tion  that  would  be  assessed  as  oscillatory  since  R<0.  For  Case  2, 
^yc-M  Is  +0.36  and  ^yM-p  Is  +0.28,  a  situation  that  would  be 
concluded  as  being  monotonically  convergent  since  0</?<l.  Fi¬ 
nally,  for  Case  3,  ^yc-M  is  +0.14  and  Ay^-f  is  +0.36,  a  situ¬ 
ation  that  would  be  concluded  as  divergent  since  /?>  1.  Thus,  for 
the  same  (true)  oscillatory  behavior  any  of  three  conclusions  can 
be  supported,  depending  on  the  relationship  of  the  chosen  grid 
size  to  the  unknown  period(s)  of  the  oscillation(s). 


Fig.  1  Illustration  of  results  of  grid  convergence  studies  using 
three  grids  when  the  true  behavior  is  oscillatory  convergence 


Table  1  Results  from  grid  convergence  studies 


Case 

Grid  Sizes 

(Ax's) 

y  Coarse 

y  Medium 

y  Fine 

Ayc-M 

AyM-F 

Behavior 

Indicadoa 

1 

2.0, 1.0, 0.5 

2.00 

3.00 

2.70 

-1.00 

0.30 

R-.0.3 

Oscillatory 

2 

0.8, 0.4, 0.2 

2.95 

2.59 

2.31 

0.36 

0.28 

R-+0.8 

Converging 

3 

1.2, 0.6, 0.3 

2.95 

2.81 

2.45 

0.14 

0.36 

R-+2.6 

Diverging 
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Note  that  in  real  cases  (with  results  from  three  grids,  say),  the 
true  behavior  of  y  with  Aa:  is  unknown,  so  the  only  information 
one  has  is  the  three  computed  values  of  y.  If  the  true  but  unknown 
behavior  is  oscillatory,  then  depending  on  the  choices  of  initial 
grid  size  and  the  grid  refinement  ratio  those  three  values  can  pro¬ 
duce  any  value  of  R  (including  a  value  indicating  asymptotic  con¬ 
vergence).  Also  note  that  whether  the  Ajc  in  one’s  finest  grid 
corresponds  to  a  value  of  1,  2,  3,  or  100  on  a  scale  such  as  that 
shown  in  Fig.  1  is  unknown. 

Conclusion 

Although  the  example  presented  is  somewhat  contrived,  the 
dilemma  one  faces  in  interpreting  results  of  grid  convergence 
studies  is  not.  If  there  is  the  possibility  of  oscillatory  behavior  of 
the  value  of  a  computed  variable  as  grid  size  is  refined  in  a  simu¬ 
lation,  then  interpretation  of  the  results  of  grid  convergence  stud¬ 
ies  seems  impossible  to  achieve  unambiguously. 
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Introduction 

Many  researchers  have  investigated  the  Fanning  factors  in  cir¬ 
cular  ducts  and  proposed  many  correlating  equations  to  calculate 
the  Fanning  factors.  As  to  various  noncircular  ducts,  the  frictional 
pressure  drops  have  rarely  been  investigated.  So  it  has  been  com¬ 
mon  practice  in  the  field  of  fluid  mechanics  to  use  the  hydraulic  or 
equivalent  diameter  in  the  Reynolds  number  in  predicting  turbu¬ 
lent  pressure  drops  along  duct  lengths  having  noncircular  cross 
section. 

But,  there  is  usually  large  deviation  from  the  circular  tube  line 
by  using  the  hydraulic  diameter  in  the  Reynolds  number.  And, 
therefore,  some  researchers  have  proposed  various  modifying 
methods  to  predict  friction  factors.  In  the  case  of  rectangular 
ducts,  for  example,  Jones  [1]  uses  a  “laminar  equivalent  diam¬ 
eter”  to  form  the  Reynolds  number,  which  is  in  turn  used  in  any 
circular  tube  correlation  for  friction  factors.  And  in  case  of  annuli, 
Brighton  and  Jones  [2]  modify  the  constant  C  in  the  Blasius  equa¬ 
tion  on  the  basis  of  the  experimental  data.  As  to  triangular  ducts. 
Nan  and  Dou  [3]  use  an  area  equivalent  round  diameter  in  the 
Reynolds  number. 
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In  the  case  of  regular  polygonal  ducts,  there  were  no  experi¬ 
mental  data  reported  in  the  literature  except  equilateral-triangular 
[4]  and  square  [5]  ducts.  Therefore,  the  purpose  of  the  investiga¬ 
tion  reported  here  was  to  obtain  friction  factors  for  isothermal, 
fully  developed,  laminar,  and  turbulent  flow  in  smooth  equilateral- 
triangular,  square,  pentagonal,  hexagonal,  heptagonal,  and  octago¬ 
nal  ducts,  respectively.  Moreover,  an  area  equivalent  round  diam¬ 
eter  is  proposed  to  use  in  Reynolds  number  in  predicting  Fanning 
factor  of  turbulent  flow  in  a  duct  having  regular  (n-sided)  polygo¬ 
nal  cross  section. 

Apparatus 

The  regular  polygonal  ducts  were  made  of  plate  glass.  Take  a 
hexagonal  duct  for  example.  First  cut  six  slabs  of  plate  glass 
(width  8.10  mm,  thickness  3.0  mm).  Then  precisely  work  pattern 
plates  of  the  hexagonal  polygon.  Put  the  slabs  of  glass  into  the 
pattern  plates.  Finally  seal  the  seams  between  the  adjacent  slabs 
with  silica  gel,  as  shown  in  Fig.  1.  The  ducts  made  of  plate  glass, 
therefore,  are  hydraulically  smooth. 

The  sketch  of  the  experimental  procedure  used  to  obtain  pres¬ 
sure  drop  data  is  illustrated  in  Fig.  2  and  Fig.  3.  In  the  setup  used, 
water  was  pumped  from  the  water  pool  into  the  elevated  tank  that 
had  an  overflow  pipe  to  maintain  a  fixed  water  level,  then  flowed 
through  a  duct  and  past  an  abrupt  entrance  and  into  the  hydrody¬ 
namic  entrance  section  where  it  became  fiilly  developed.  It  then 
entered  the  test  section  that  consisted  of  four  identical  test  sections 
in  series.  These  multiple  section  served  to  check  on  the  reproduc¬ 
ibility  of  the  measurements  and  to  insure  that  the  flow  was  fully 
developed.  In  the  test  section,  the  pressure  drop  readings  were 
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Fig.  1  Structure  of  the  hexagonal  polygonal  duct 
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Fig.  3  Sketch  of  the  positions  of  the  pressure  holes  distribut¬ 
ing  over  the  test  sections  (TS) 
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Part  1  of  this  two-part  paper  presents  a  comprehensive  approach  to  verification  and 
validation  methodology  and  procedures  for  CFD  simulations  from  an  already  developed 
CFD  code  applied  without  requiring  availability  of  the  source  code  for  specified  objec¬ 
tives,  geometry,  conditions,  and  available  benchmark  information.  Concepts,  definitions, 
and  equations  derived  for  simulation  errors  and  uncertainties  provide  the  overall  math¬ 
ematical  framework  Verification  is  defined  as  a  process  for  assessing  simulation  numeri¬ 
cal  uncertainty  and,  when  conditions  permit,  estimating  the  sign  and  magnitude  of  the 
numerical  error  itself  and  the  uncertainty  in  that  error  estimate.  The  approach  for  esti¬ 
mating  enors  and  uncertainties  includes  (1)  the  option  of  treating  the  numerical  error  as 
deterministic  or  stochastic,  (2)  the  use  of  generalized  Richardson  extrapolation  for  J  input 
parameters,  and  (3)  the  concept  of  correction  factors  based  on  analytical  benchmarks, 
which  provides  a  quantitative  metric  to  determine  proximity  of  the  solutions  to  the 
asymptotic  range,  accounts  for  the  effects  of  higher-order  terms,  and  are  used  for  defining 
and  estimating  errors  and  uncertainties.  Validation  is  defined  as  a  process  for  assessing 
simulation  modeling  uncertainty  by  using  benchmark  experimental  data  and,  when  con¬ 
ditions  permit,  estimating  the  sign  and  magnitude  of  the  modeling  error  itself  The  ap¬ 
proach  properly  takes  into  account  the  uncertainties  in  both  the  simulation  and  experi¬ 
mental  data  in  assessing  the  level  of  validation.  Interpretation  of  results  of  validation 
efforts  both  where  the  numerical  error  is  treated  as  deterministic  and  stochastic  are 
discussed.  Part  2  provides  an  example  for  PANS  simulations  for  a  cargo/container  ship 
where  issues  with  regard  to  practical  application  of  the  methodology  and  procedures  and 
interpretation  of  verification  and  validation  results  are  discussed. 

[DOI:  10.1115/1.1412235] 


1  Introduction 

Discussion  and  methodology  for  estimating  errors  and  uncer¬ 
tainties  in  computational  fluid  dynamics  (CFD)  simulations  have 
reached  a  certain  level  of  maturity  with  recognition  of  importance 
through  editorial  policies  (Freitas  [1]),  increased  attention  and  re¬ 
cent  progress  on  common  terminology  (AIAA,  [2]),  advocacy  and 
detailed  methodology  (Roache  [3]),  and  numerous  case  studies 
(e.g.  [4]).  Progress  has  been  accelerated  in  response  to  the  urgent 
need  for  achieving  consensus  on  concepts,  definitions,  and  useful 
methodology  and  procedures,  as  CFD  is  applied  to  increasingly 
complex  geometry  and  physics  and  integrated  into  the  engineering 
design  process.  Such  consensus  is  required  to  realize  the  goals  of 
simulation-based  design  and  other  uses  of  CFD  such  as  simulating 
flows  for  which  experiments  are  difficult  (e.g.,  full-scale  Reynolds 
numbers,  hypersonic  flows,  off-design  conditions).  In  spite  of  the 
progress  and  urgency,  the  various  viewpoints  have  not  converged 
and  current  approaches  fall  short  of  providing  practical  methodol¬ 
ogy  and  procedures  for  estimating  errors  and  uncertainties  in  CFD 
simulations. 

The  present  work  provides  a  pragmatic  approach  for  estimating 
errors  and  uncertainties  in  CFD  simulations.  Previous  work  on 
verification  (Stem  et  al.  [5])  is  extended  and  put  on  a  more  rigor¬ 
ous  foundation  and  combined  with  subsequent  work  on  validation 
(Coleman  and  Stem  [6])  thereby  providing  a  comprehensive 
framework  for  overall  procedures  and  methodology.  The  philoso- 
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phy  is  strongly  influenced  by  experimental  fluid  dynamics  (EFD) 
uncertainty  analysis  (Coleman  and  Steele  [7]),  which  has  been 
standardized.  Hopefully,  CFD  verification  and  validation  proce¬ 
dures  and  methodology  can  reach  a  similar  level  of  maturity  and 
user  variability  can  reach  similar  low  levels,  as  for  EFD.  The  work 
is  part  of  a  larger  program  (Rood  [8])  for  developing  and  imple¬ 
menting  a  strategy  for  verification  and  validation  of  Reynolds- 
averaged  Navier-Stokes  (RANS)  ship  hydrodynamics  CFD  codes. 
The  program  includes  complementary  CFD  and  EFD  towing-tank 
investigations  and  considers  errors  and  uncertainties  in  both  the 
simulations  and  the  data  in  assessing  the  success  of  the  verifica¬ 
tion  and  validation  efforts.  The  work  also  benefited  from  collabo¬ 
ration  with  the  21®*  and  22"^^  International  Towing  Tank  Resistance 
Committees  (ITTC  [9,10]).  The  procedures  proposed  in  this  paper 
were  adopted  on  an  interim  basis  by  the  22"*^  ITTC  and  also  were 
recommended  and  used  at  the  recent  Gothenburg  2(X)0  Workshop 
on  CFD  in  Ship  Hydrodynamics  (Larsson  et  al.  [11]). 

The  focus  is  on  verification  and  validation  methodology  and 
procedures  for  CFD  simulations  with  an  already  developed  CFD 
code  applied  without  requiring  availability  of  the  source  code  for 
specified  objectives,  geometry,  conditions,  and  available  bench¬ 
mark  information.  The  methodology  and  procedures  were  devel¬ 
oped  considering  RANS  CFD  codes,  but  should  be  applicable  to  a 
fairly  broad  range  of  codes  such  as  boundary-element  methods 
and  certain  aspects  of  large-eddy  and  direct  numerical  simula¬ 
tions.  The  present  work  differs  in  many  respects  from  recent  lit¬ 
erature.  The  presentation  is  relatively  succinct  with  intention  for 
use  for  practical  applications  (i.e.,  industrial  CFD)  for  which  nu¬ 
merical  errors  and  uncertainties  cannot  be  considered  negligible  or 
overlooked. 
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The  definitions  of  errors  and  uncertainties  and  verification  and 
validation  that  are  used  in  any  approach  need  to  be  clearly  stated. 
The  present  and  Roache  [3]  definitions  for  errors  and  uncertainties 
are  consistent  with  those  used  for  EFD.  The  AIAA  [2]  definitions 
are  from  an  information  theory  perspective  and  differ  from  those 
used  in  EFD»  but  are  not  contradictory  to  the  present  definitions. 
The  present  concepts  and  definitions  for  verification  and  valida¬ 
tion  are  closely  tied  to  the  present  definitions  of  errors  and  uncer¬ 
tainties  and  equations  derived  for  simulation  errors  and  uncertain¬ 
ties  thereby  providing  the  overall  mathematical  framework.  The 
Roache  [3]  and  AIAA  [2]  definitions  are  broader,  but  not  contra¬ 
dictory  to  the  present  definitions.  The  present  approach  includes 
both  the  situations  (1)  of  estimating  errors  and  the  uncertainty  of 
those  estimates  and  (2)  of  estimating  uncertainties  only.  Richard¬ 
son  extrapolation  (RE)  is  used  for  verification,  which  is  not  new; 
however,  the  present  generalizations  for  J  input  parameters  and 
concept  of  correction  factors  based  on  analytic^  benchmarks, 
which  provides  a  quantitative  metric  to  determine  proximity  of  the 
solutions  to  the  asymptotic  range,  accounts  for  the  effects  of 
higher-order  terms,  and  are  used  for  defining  and  estimating  errors 
and  uncertainties  constitute  a  new  approach.  The  use  of  quantita¬ 
tive  estimates  for  errors  and  the  use  of  uncertainties  for  those 
estimates  also  constitute  a  new  approach  in  verification  and 
validation. 

Part  1  of  this  two-part  paper  presents  the  verification  and  vali¬ 
dation  methodology  and  procedures.  In  Section  2,  the  overall  veri¬ 
fication  and  validation  methodology  is  presented  by  providing 
concepts,  definitions,  and  equations  for  the  simulation  numerical 
and  modeling  errors  and  uncertainties.  In  Section  3,  detailed  veri¬ 
fication  procedures  for  estimation  of  various  sub-components  of 
the  simulation  numerical  error  and  uncertainty  are  given.  In  Sec¬ 
tion  4,  validation  procedures  are  given  including  a  discussion  of 
the  interpretation  of  validation  results  and  use  of  corrected  simu¬ 
lation  results.  Finally,  conclusions  are  provided  in  Section  5.  Part 
2  provides  an  example  for  RANS  simulations  for  a  cargo/ 
container  ship  where  issues  with  regard  to  practical  application  of 
the  methodology  and  procedures  and  interpretation  of  verification 
and  validation  results  are  discussed  (Wilson  et  al.  [12]).  Present 
papers  are  based  on  Stem  et  al.  [13],  which  is  sometimes  refer¬ 
enced  for  additional  details.  However,  presentation  and  expanded 
discussions  of  verification  procedures  and  implementation  were 
improved  based  on  nearly  two  years  experience  with  present  ap¬ 
proach,  especially  through  ITTC  community  and  Gothenburg 
2000  Workshop  on  CFD  in  Ship  Hydrodynamics. 

2  Overall  Verification  and  Validation  Methodology 

In  Section  2.1,  the  overall  verification  and  validation  method¬ 
ology  is  presented  by  providing  key  concepts,  definitions,  and 
derivation  of  equations  for  the  simulation  error  and  uncertainty,  as 
sum  and  root-sum-square  (RSS)  of  simulation  numerical  and 
modeling  errors  and  uncertainties,  respectively.  The  verification 
and  validation  equations  are  derived  in  Sections  2.2  and  2.3,  re¬ 
spectively,  where  subcomponents  of  the  simulation  numerical  er¬ 
ror  are  identified  and  an  approach  for 'assessing  the  simulation 
modeling  uncertainty  is  presented. 

2.1  Concepts  and  I>efinitions.  Accuracy  indicates  the 
closeness  of  agreement  between  a  simulation/experimental  value 
of  a  quantity  and  its  true  value.  Error  6  is  the  difference  between 
a  simulation  value  or  an  experimental  value  and  the  trath.  Accu¬ 
racy  increases  as  error  approaches  zero.  The  true  values  of 
simulation/experimental  quantities  are  rarely  known.  Thus,  errors 
must  be  estimated.  An  uncertainty  U  is  m  estimate  of  an  error 
such  that  the  interval  ±  U  contains  the  true  value  of  S  95  times 
out  of  100.  An  uncertainty  interval  thus  indicates  the  range  of 
likely  magnitudes  of  S  but  no  information  about  its  sign. 

For  simulations,  under  certain  conditions,  errors  can  be  esti¬ 
mated  including  both  sign  and  magnitude  (referred  to  as  an  error 
estimate  <?*).  Then,  the  uncertainty  considered  is  that  correspond¬ 


ing  to  the  error  in  When  S*  is  estimated,  it  can  be  used  to 
obtain  a  corrected  value  of  the  variable  of  interest. 

Sources  of  errors  and  uncertainties  in  results  from  simulations 
can  be  divided  into  two  distinct  sources:  modeling  and  numerical. 
Modeling  errors  and  uncertainties  are  due  to  assumptions  and  ap¬ 
proximations  in  the  mathematical  representation  of  the  physical 
problem  (such  as  geometry,  mathematical  equation,  coordinate 
transformation,  boundary  conditions,  turbulence  models,  etc.)  and 
incorporation  of  previous  data  (such  as  fluid  properties)  into  the 
model.  Numericsd  errors  and  uncertainties  are  due  to  numerical 
solution  of  the  mathematical  equations  (such  as  discretization,  ar¬ 
tificial  dissipation,  incomplete  iterative  and  grid  convergence,  lack 
of  conservation  of  mass,  momentum,  and  energy,  internal  and  ex¬ 
ternal  boundary  noncontinuity,  computer  round-off,  etc.).  The 
present  work  assumes  that  all  correlations  among  errors  are  zero, 
which  is  doubtless  not  true  in  all  cases,  but  the  effects  are  assumed 
negligible  for  the  present  analyses. 

The  simulation  error  Ss  is  defined  as  the  difference  between  a 
simulation  result  S  and  the  truth  T.  In  considering  the  develop¬ 
ment  and  execution  of  a  CFD  code,  it  can  be  postulated  that  ^5  is 
comprised  of  the  addition  of  modeling  and  numerical  errors 

Ss~S—T~SsM'^Ssf^  (1) 

Support  for  this  postulation  is  provided  by  using  the  model  value 
M  in  definitions  for  modeling  and  numerical  errors.  The  simula¬ 
tion  modeling  error  ^SM  is  defined  as  the  difference  be¬ 

tween  the  true  T  and  model  M  values  while  the  simulation  numeri¬ 
cal  error  S^n^S-M  is  defined  as  the  difference  between  the 
simulation  S  and  model  M  values.  The  simulation  S  and  model  M 
values  are  obtained  by  numerical  and  exact  solutions  of  the  con¬ 
tinuous  equations  used  to  model  the  truth,  respectively.  Since  ex¬ 
act  solution  of  nonlinear  equations  is  seldom  possible,  approxima¬ 
tions  are  used  to  replace  the  continuous  modeled  equations  with 
discrete  ones  that  are  solved  algebraically  with  a  CFD  code  to 
yield  the  simulation  value  5. 

The  uncertainty  equation  corresponding  to  error  equation  (1)  is 

where  I/5  is  the  uncertainty  in  the  simulation  and  Vsm  and  Usn 
are  the  simulation  modeling  and  numerical  uncertainties. 

For  certain  conditions,  the  numerical  error  S^n  can  be  consid¬ 
ered  as 

(3) 

where  ^5^  is  an  estimate  of  the  sign  and  magnitude  of  Ssn  and 
BsN  is  the  error  in  that  estimate  (and  is  estimated  as  an  uncertainty 
since  only  a  range  bounding  its  magnitude  and  not  its  sign  can  be 
estimated).  The  corrected  simulation  value  Sc  is  defined  by 

(4) 

with  error  equation 

(5) 

The  uncertainty  equation  corresponding  to  error  equation  (5)  is 

where  U 5^  is  the  uncertainty  in  the  corrected  simulation  and  1/5^ 
is  the  uncertainty  estimate  for  e  sh  • 

Debate  on  verification  and  validation  has  included  discussion 
on  whether  errors  such  as  are  deterministic  or  stochastic,  and 
thus  how  they  should  be  treated  in  uncertainty  analysis  was  un¬ 
clear.  In  the  “corrected”  approach  given  by  Eqs.  (3)-(6),  a  deter¬ 
ministic  estimate  of  and  consideration  of  the  error  65^  in 
that  estimate  are  used.  The  approach  is  analogous  to  that  in  EFD 
when  an  asymmetric  systematic  uncertainty  is  “zero-centered”  by 
inclusion  of  a  model  for  the  systematic  error  in  the  data  reduction 
equation  and  then  the  uncertainty  considered  is  that  associated 
with  the  model  (Coleman  and  Steele  [7]).  In  the  “uncorrected” 
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approach  given  by  Eqs.  (l)-(2),  any  particular  Ssi^  is  considered 
as  a  single  realization  from  some  parent  population  of  and 
the  uncertainty  Usn  is  interpreted  accordingly  in  analogy  to  the 
estimation  of  uncertainties  in  EFD  (with  a  similar  argument  for 
Bsn  and  Oberkamp  and  Trucano  [14]  have  criticized  Cole¬ 
man  and  Stem  [6]  for  treating  Usi^  statistically;  however,  the 
present  approach  is  well  justified  both  conceptually  and  math¬ 
ematically  for  reasons  just  given. 

The  overall  CFD  verification  and  validation  procedures  can  be 
conveniently  grouped  in  four  consecutive  steps.  The  first  step  is 
preparation,  which  involves  selection  of  the  CFD  code  and  speci¬ 
fication  of  objectives,  geometry,  conditions,  and  available  bench¬ 
mark  information.  The  objectives  might  be  prediction  of  certain 
variables  at  certain  levels  of  validation  (e.g.,  programmatic  vali¬ 
dation  requirements  U^egd)-  The  variables  can  either  be  integral 
(e.g.,  resistance)  or  point  (e.g.,  mean  velocities  and  turbulent  Rey¬ 
nolds  stresses)  values  and  the  programmatic  validation  require¬ 
ments  may  be  different  for  each  variable.  The  second  and  third 
steps  are  verification  and  validation,  which  are  described  in  Sec¬ 
tions  2.2  and  2.3.  The  fourth  step  is  documentation,  which  is 
detailed  presentation  of  the  CFD  code  (equations,  initial  and 
boundary  conditions,  modeling,  and  numerical  methods),  objec¬ 
tives,  geometry,  conditions,  verification,  validation,  and  analysis. 

2.2  Verification.  Verification  is  defined  as  a  process  for  as¬ 
sessing  simulation  numerical  uncertainty  Usfsf  and,  when  condi¬ 
tions  permit,  estimating  the  sign  and  magnitude  of  the  simu¬ 
lation  numerical  error  itself  and  the  uncertainty  in  that  error 
estimate.  For  many  CFD  codes,  the  most  important  numerical 
errors  and  uncertainties  are  due  to  use  of  iterative  solution  meth¬ 
ods  and  specification  of  various  input  parameters  such  as  spatial 
and  time  step  sizes  and  other  parameters  (e.g.,  artificial  dissipa¬ 
tion).  The  errors  and  uncertainties  are  highly  dependent  on  the 
specific  application  (geometry  and  conditions). 

The  errors  due  to  specification  of  input  parameters  are  decom¬ 
posed  into  error  contributions  from  iteration  number  Sj ,  grid  size 
Sq,  time  step  and  other  parameters  Sp,  which  gives  the 
following  expressions  for  the  simulation  numerical  error  and  un¬ 
certainty 


j 


(7) 

J 

(8) 

Similarly,  error  estimates  <5*  can  be  decomposed  as 

J 

7=1 

(9) 

which  gives  the  following  expressions  for  the  corrected  simulation 
and  corrected  simulation  numerical  uncertainty 

Sc=S-|<5*  +  2  <5;|=r+<5,j^+e5^ 

(10) 

(11) 

Verification  is  based  on  equation  (10),  which  is  put  in  the  form 


(12) 

Equation  (12)  expresses  5  as  the  corrected  simulation  value  Sc 
plus  numerical  errors.  5^  is  also  referred  to  as  a  numerical  bench¬ 


Flg.  1  Definition  of  comparison  error. 


mark  since  it  is  equal,  as  shown  by  Eq.  (10),  to  the  truth  plus 
simulation  modeling  error  and  presumable  small  error  e  sn  Ih® 
estimate  of  the  numerical  error  . 

23  Validation.  Validation  is  defined  as  a  process  for  assess¬ 
ing  simulation  modeling  uncertainty  Usm  by  using  benchmark 
experimental  data  and,  when  conditions  permit,  estimating  the 
sign  and  magnitude  of  the  modeling  error  itself.  Thus,  the 
errors  and  uncertainties  in  the  experimental  data  must  be  consid¬ 
ered  in  addition  to  the  numerical  errors  and  uncertainties  dis¬ 
cussed  in  Section  3.  Approaches  to  estimating  experimental  un¬ 
certainties  are  presented  and  discussed  by  Coleman  and  Steele  [7]. 

The  validation  methodology  of  Coleman  and  Stem  [6]  which 
properly  takes  into  account  the  uncertainties  in  both  the  simula¬ 
tion  and  the  experimental  data  is  discussed  in  this  section  for  both 
approaches  of  treating  the  numerical  error  as  stochastic  and  as 
deterministic. 

The  validation  comparison  is  shown  in  Fig.  1.  The  experimen¬ 
tally  determined  r- value  of  the  (X/  ,rf)  data  point  is  D  and  simu¬ 
lated  r-value  is  5.  Recall  from  Eq.  (1)  that  the  simulation  error  Ss 
is  the  difference  between  S  and  the  truth  T.  Similarly,  the  error  Sq 
in  the  data  is  the  difference  between  D  and  the  truth  T,  so  setting 
the  simulation  and  experimental  truths  equal  results  in 

D-So=S-Ss  (13) 

The  comparison  error  E  is  defined  as  the  difference  of  D  and  S 


E  —  D  S—S[)  ^3"“  ^SPD'^  ^Sn)  (14) 

with  SsM  decomposed  into  the  sum  of  Sspd  .  error  from  the  use  of 
previous  data  such  as  fluid  properties,  and  ,  error  from  mod¬ 
eling  assumptions.  Thus  E  is  the  resultant  of  all  the  errors  asso¬ 
ciated  both  with  the  experimental  data  and  with  the  simulation. 
For  the  approach  in  which  no  estimate  of  the  sign  and  mag¬ 
nitude  of  Sss  is  made,  all  of  these  errors  are  estimated  with  un¬ 
certainties. 

If  (X/,r;),  and  S  share  no  conunon  error  sources,  then  the  un¬ 
certainty  C/f  in  the  comparison  error  can  be  expressed  as 


or 


—\ 

ss) 


+u]=ul+ul 


v\—  t/o+  ^\n 


(15) 

(16) 


Ideally,  one  would  postulate  that  if  the  absolute  value  of  E  is 
less  than  its  uncertainty  then  validation  is  achieved  (i.e.,  £is 
“zero’*  considering  the  resolution  imposed  by  the  “noise  level’* 
Ue).  In  reality,  there  is  no  known  approach  that  gives  an  estimate 
of  ^sMA  ♦  so  t/£  cannot  be  estimated.  That  leaves  a  more  stringent 
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validation  test  as  the  practical  alternative.  If  the  validation  uncer¬ 
tainty  Uyis  defined  as  the  combination  of  all  uncertainties  that  we 
know  how  to  estimate  (i.e.,  all  but  Usma)^ 

Ul=U\-Vl^^  =  Ul+U\rE  (17) 

where  is  the  total  estimated  simulation  uncer¬ 

tainty,  as  shown  in  Fig.  1. 

If  l£|  is  less  than  the  validation  uncertainty  combina¬ 

tion  of  all  the  errors  in  D  and  S  is  smaller  than  the  estimated 
validation  uncertainty  and  validation  has  been  achieved  at  the  f/y 
level,  f/y  is  the  key  metric  in  the  validation  process,  t/y  is  the 
validation  “noise  level”  imposed  by  the  uncertainties  inherent  in 
the  data,  the  numerical  solution,  and  the  previous  experimental 
data  used  in  the  simulation  model.  It  can  be  argued  that  one  can¬ 
not  discriminate  once  \E\  is  less  than  this;  that  is,  as  long  as  |£|  is 
less  than  this,  one  cannot  evaluate  the  effectiveness  of  proposed 
model  “improvements.”  On  the  other  hand,  if  \E\>Uy  one  could 
argue  that  probably  Ssma  • 

Oberkamp  and  Trucano  [14]  have  criticized  Coleman  and  Stem 
[6]  for  fact  that  Uy  excludes  Usma  •  As  already  acknowledged, 
there  is  no  known  way  for  directly  estimating  Vsma  •  However, 
the  present  approach  does  provide  a  more  stringent  validation 
metric  U  y  which  sets  the  level  that  validation  can  be  achieved  as 
the  root  sum  square  of  the  experimental  Uq  and  the  total  esti¬ 
mated  simulation  Uste  uncertainties.  Additionally,  under  certain 
conditions,  the  simulation  modeling  error  S^ma  ^Iself  can  be  esti¬ 
mated,  as  further  discussed  in  Section  4.  Consideration  of  Eq.  (17) 
shows  that  (1)  the  more  uncertain  the  data  (greater  V^)  and/or  (2) 
the  more  inaccurate  the  code  (greater  Ust^,  the  easier  it  is  to 
validate  a  code,  since  the  greater  the  uncertainties  in  the  data  and 
code  predictions,  the  greater  the  noise  level  U y .  Both  Roache  [3] 
and  Oberkamp  and  Trucano  [14]  have  criticized  Coleman  and 
Stem  [6]  for  this  fact.  However,  if  the  value  of  f/y  is  greater  than 
that  designated  as  necessary  in  a  research/design/development 
program,  the  required  level  of  validation  could  not  be  achieved 
without  improvement  in  the  quality  of  the  data,  the  code,  or  both. 
Also,  if  UsN  and  U^pd  are  not  estimated,  but  |£|  is  less  than  Up, 
then  a  type  of  validation  can  be  argued  to  have  been  achieved,  but 
clearly  as  shown  by  the  present  methodology,  at  an  unknown 
level. 

If  the  “corrected”  approach  of  Eqs.  (3) -(6)  is  used,  then  the 
equations,  equivalent  to  Eqs.  (14)  and  (17)  are 

Ec—D’-Sc=Sp—{Ssma'^^spd^^sn)  (18) 

for  the  corrected  comparison  error  and 

Ul=  Ul-  =  ul=  Ui+  Ul^rE  (19) 

for  the  corrected  validation  uncertainty  where  Ug^pp—Ulpp 
+  is  the  total  estimated  corrected  simulation  uncertainty, 
also  shown  in  Fig.  1.  Note  that  Sc  and  Ec  can  be  either  larger  or 
smaller  than  their  counterparts  S  and  £,  but  U and  U y^  should 
be  smaller  than  Up  and  Uy,  respectively,  since  Us^n  should  be 
smaller  than  U sn  • 

If  there  is  a  programmatic  validation  requirement,  there  is  an¬ 
other  uncertainty  U^eg^  that  must  be  considered  since  validation  is 
required  at  that  uncertainty  level  or  below.  Interpretation  of  the 
meaning  of  the  relative  magnitudes  of  E  (or  Ec),  Ureqd 
(or  U  yp  and  of  the  implications  on  the  possibility  of  estimating 
8sma  discussed  in  Section  4.  Additional  discussion  is  provided 
in  Coleman  and  Stem  [6]  on:  estimating  Uspo'^  estimating  Up  for 
the  data  point  (X,-,r/),  including  both  the  experimental  uncer¬ 
tainty  in  T/  and  the  additional  uncertainties  in  arising  from  the 
experimental  uncertainties  in  the  measurements  of  the  n  indepen¬ 
dent  variables  (Xj)i  in  X,- ;  and  for  validation  of  a  CFD  code, 
multiple  codes  and/or  models,  and  prediction  of  trends. 


3  Verification  Procedures 

In  Section  2,  the  simulation  numerical  error  and  uncertainty 
were  decomposed  into  contributions  from  iteration  number,  grid 
size,  time  step,  and  other  parameters  in  Eqs.  (7)  and  (8).  In  this 
section,  detailed  verification  procedures  are  given  for  estimation 
of  these  contributions  through  convergence  studies  (Section  3,1). 
Iterative  (Section  3.2)  and  parameter  (Sections  3.3-3.5)  conver¬ 
gence  studies  are  conducted  using  multiple  solutions  with  system¬ 
atic  parameter  refinement  to  estimate  numerical  errors  and  uncer¬ 
tainties.  Three' convergence  conditions  are  possible:  (/)  monotonic 
convergence;  (if)  oscillatory  convergence;  and  (Hi)  divergence  and 
are  described  in  Sections  3.3,  3.4,  and  3.5,  respectively.  For  con¬ 
dition  (f),  as  already  mentioned,  errors  and  uncertainties  are  esti¬ 
mated  using  generdized  RE.  For  condition  (fi),  uncertainties  are 
estimated  simply  by  attempting  to  bound  error  based  on  oscilla¬ 
tion  maximums  and  minimums.  For  condition  (///)♦  errors  and  un¬ 
certainties  cannot  be  estimated.  As  discussed  below  and  later  in 
Section  5,  there  are  many  issues  in  estimating  errors  and  uncer¬ 
tainties  for  practical  applications. 

3.1  Convergence  Studies.  Iterative  and  parameter  conver¬ 
gence  studies  are  conducted  using  multiple  (m)  solutions  and  sys¬ 
tematic  parameter  refinement  by  varying  the  ^:th  input  parameter 
AjC|t  while  holding  all  other  parameters  constant.  The  present  work 
assumes  input  parameters  can  be  expressed  such  that  the  finest 
resolution  corresponds  to  the  limit  of  infinitely  small  parameter 
values.  Many  common  input  parameters  are  of  this  form,  e.g.,  grid 
spacing,  time  step,  and  artificial  dissipation.  Additionally,  a  uni¬ 
form  parameter  refinement  ratio 

=  AjCfc^/AjC)t^  j  between  solutions  is  assumed  for  presentation 
purposes,  but  not  required  as  discussed  later. 

Careful  consideration  should  be  given  to  selection  of  uniform 
parameter  refinement  ratio.  The  most  appropriate  values  for  indus¬ 
trial  CFD  are  not  yet  fully  established.  Small  values  (i.e.,  very 
close  to  one)  are  undesirable  since  solution  changes  will  be  small 
and  sensitivity  to  input  parameter  may  be  difficult  to  identify  com¬ 
pared  to  iterative  errors.  Large  values  alleviate  this  problem;  how¬ 
ever,  they  also  may  be  undesirable  since  the  finest  step  size  may 
be  prohibitively  small  (i.e.,  require  many  steps)  if  the  coarsest  step 
size  is  designed  for  sufficient  resolution  such  that  similar  physics 
are  resolved  for  all  m  solutions.  Also,  similarly  as  for  small  val¬ 
ues,  solution  changes  for  the  finest  step  size  may  be  difficult  to 
identify  compared  to  iterative  errors  since  iterative  convergence  is 
more  difficult  for  small  step  size.  Another  issue  is  that  for  param¬ 
eter  refinement  ratio  other  than  r*  =  2,  interpolation  to  a  common 
location  is  required  to  compute  solution  changes,  which  intro¬ 
duces  interpolation  errors.  Roache  [3]  discusses  methods  for 
evaluating  interpolation  errors.  However,  for  industrial  CFD, 

=  2  may  often  be  too  large.  A  good  alternative  may  be  rf^=Vi,  as 
it  provides  fairly  large  parameter  refinement  ratio  and  at  least 
enables  prolongation  of  the  coarse-parameter  solution  as  an  initial 
guess  for  the  fine-parameter  solution. 

Equation  (12)  is  written  for  the  kth  parameter  and  mth  solution 
as 

/ 

S,=Sc+Sf+St-^  2  SI  (20) 

Iterative  convergence  must  be  assessed  and  corrected  for  it¬ 
erative  errors  prior  to  evaluation  of  parameter  convergence  since 
the  level  of  iterative  convergence  may  not  be  the  same  for  all  m 
solutions  used  in  the  parameter  convergence  studies.  Equation 
(20)  shows  that  iterative  errors  must  be  accurately  estimated 

or  negligible  in  comparison  to  8^  for  accurate  convergence  stud¬ 
ies  and  that  they  should  be  considered  within  the  context  of  con¬ 
vergence  studies  for  each  input  parameter.  Methods  for  estimating 
Uj  OT  8f  and  U are  described  in  Section  3.2.2. 

With  8f  evaluated.  Si  is  corrected  for  iterative  errors  as 

‘I  ’ 
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gence  study  on  the  amplitude  distribution  of  the  Fourier  modes.  In 
(21)  principle,  this  approach  would  remove  the  problem  of  ill- 
conditioning  of  the  convergence  ratio,  . 


Sk  can  be  calculated  for  both  integral  (e.g.,  resistance  coeffi¬ 
cients)  and  point  (e.g.,  surface  pressure,  wall-shear  stress,  and 
velocity)  variables.  can  be  presented  as  an  absolute  quantity 
(i.e.,  non-normalized)  or  normalized  with  the  solution  as  a  per¬ 
centage  change;  however,  if  the  solution  value  is  small,  a  more 
appropriate  normalization  may  be  the  range  of  the  solution. 

Convergence  studies  require  a  minimum  of  m  =  3  solutions  to 
evaluate  convergence  with  respect  to  input  parameter.  Note  that 
m  =  2  is  inadequate,  as  it  only  indicates  sensitivity  and  not  con¬ 
vergence,  and  that  m>3  may  be  required.  Consider  the  situation 
for  3  solutions  corresponding  to  fine  5^^,  medium  5*^,  and  coarse 
values  for  the  kth  input  parameter.  Solution  changes  e  for 
medium-fine  and  coarse-medium  solutions  and  their  ratio  /?jt  are 
defined  by 


3.2  Iterative  Convergence.  The  number  of  order  magni¬ 
tude  drop  and  final  level  of  solution  residual  (or  residual  imbal¬ 
ance)  can  be  used  to  determine  stopping  criteria  for  iterative  so¬ 
lution  techniques.  Iterative  convergence  to  machine  zero  is 
desirable,  but  for  complex  geometry  and  conditions  it  is  often  not 
possible.  Three  or  four  orders  of  magnitude  drop  in  solution  re¬ 
sidual  to  a  level  of  10”"^  is  more  likely  for  these  cases.  Methods 
for  estimation  of  iterative  errors  and  uncertainties  can  be  based  on 
graphical,  as  discussed  below,  or  theoretical  approaches  and  are 
dependent  on  the  type  of  iterative  convergence:  (a)  oscillatory;  (b) 
convergent;  or  (c)  mixed  oscillatory/convergent. 

For  oscillatory  iterative  convergence  (a),  the  deviation  of  the 
variable  from  its  mean  value  provides  estimates  of  the  iterative 
uncertainty  based  on  the  range  of  the  maximum  5^/  and  minimum 
Si  values 


Rlr—Bk 

K  *21  *32 

Three  convergence  conditions  are  possible: 


(22) 


t/;= 


|-(5j/— 5/^) 


(24) 


For  convergent  iterative  convergence  {b),  a  curve-fit  of  an  ex¬ 
ponential  function  can  be  used  to  estimate  Uj  ox  Sf  and  U as 
the  difference  between  the  value  and  the  exponential  function 
from  a  curve  fit  for  large  iteration  number  CF* 


(i)  Monotonic  convergence:  0</?jt<l 


t/;=|5-CFool 


(//)  Oscillatory  convergence:  Rk<0^  (23) 


Sf  =5-CF„,t//  =0  (25) 


(ill)  Divergence:  Rk>  1 

For  monotonic  convergence  (i),  generalized  RE  is  used  to  esti¬ 
mate  Uk  or  Sf  and  Methods  for  estimating  errors  and  un¬ 
certainties  for  condition  (i)  are  described  in  Section  3.3. 

For  oscillatory  convergence  (ii),  the  solutions  exhibit  oscilla¬ 
tions,  which  may  be  erroneously  identified  as  condition  (i)  or  (iii). 
This  is  apparent  if  one  considers  evaluating  convergence  condi¬ 
tion  from  three  points  on  a  sinusoidal  curve  (Coleman  et  al.  [15]). 
Depending  on  where  the  three  points  fall  on  the  curve,  the  condi¬ 
tion  could  be  incorrectly  diagnosed  as  either  monotonic  conver¬ 
gence  or  divergence.  Methods  discussed  here  for  estimating  un¬ 
certainties  Uk  for  condition  (ii)  require  more  than  m  =  3  solutions 
and  are  described  in  Section  3.4. 

For  divergence  (iii),  the  solutions  diverge  and  errors  and  uncer¬ 
tainties  cannot  be  estimated.  Additional  remarks  are  given  in  Sec¬ 
tion  3.5. 

Determination  of  the  convergence  ratio  R^  for  point  variables 
can  be  problematic  since  solution  changes 
go  to  zero  (e.g.,  in  regions  where  the  solution  contains  an  inflec¬ 
tion  point).  In  this  case,  the  ratio  becomes  ill  conditioned.  How¬ 
ever,  the  convergence  ratio  can  be  used  in  regions  where  the  so¬ 
lution  changes  are  both  non-zero  (e.g.,  local  solution  maximums 
or  minimums).  Another  approach  is  to  use  a  global  convergence 
ratio  Rk ,  which  overcomes  ill  conditioning,  based  on  the  L2  norm 
of  the  solution  changes,  i.e.,  (F*)  =  116*2^12 /||e*J|2.  (>  is  used  to 

denote  an  averaged  value  and  |lell2=[2fLi£?]*^  denotes  the  L2 
norm  of  solution  change  over  the  N  points  in  the  region  of  inter¬ 
est.  Caution  should  be  exercised  when  defining  the  convergence 
ratio  from  the  ratio  of  the  L2  norm  of  solution  changes  because 
the  oscillatory  condition  (F*<1)  cannot  be  diagnosed  since  (Rk) 
will  always  be  greater  than  zero.  Local  values  of  Rk  at  solution 
maximums  or  minimums  should  also  be  examined  to  confirm  the 
convergence  condition  based  on  an  L2  norm  definition.  An  alter¬ 
nate  approach  suggested  by  Hoekstra  et  al.  [16]  is  to  transform  the 
spatial  profile  to  wave  number  space  and  to  perform  a  conver- 

‘As  discussed  in  the  text  that  follows,  0</?t<  1  and  /?*>  1  may  also  occur  for 
the  oscillatory  condition. 


For  mixed  convergent/oscillatory  iterative  convergence  (c),  the 
amplitude  of  the  solution  envelope  decreases  as  the  iteration  num¬ 
ber  increases,  the  solution  envelope  is  used  to  define  the  maxi¬ 
mum  Sy  and  minimum  Si  values  in  the  Ith  iteration,  and  to  esti¬ 
mate  Uj  ox  Sf  and 


2(5</-St) 


(26) 


Sf^^=S-j(Su-S0,U,^^Q 


An  increase  in  the  amplitude  of  the  solution  envelope  as  the 
iteration  number  increases  indicates  that  the  solution  is  divergent. 

Estimates  of  the  iterative  error  based  on  theoretical  approaches 
are  presented  in  Ferziger  and  Peric  [17]  and  involve  estimation  of 
the  principal  eigenvalue  of  the  iteration  matrix.  The  approach  is 
relatively  straightforward  when  the  eigenvalue  is  real  and  the  so¬ 
lution  is  convergent.  For  cases  in  which  the  principal  eigenvalue  is 
complex  and  the  solution  is  oscillatory  or  mixed,  the  estimation  is 
not  as  straightforward  and  additional  assumptions  are  required. 


3.3  Monotonic  Convergence:  Generalized  Richardson  Ex¬ 
trapolation.  For  monotonic  convergence,  i.e.,  condition  (i)  in 
Eq.  (23),  generalized  RE  is  used  to  estimate  Uk  ox  Sf  and  C/jt^. 
RE  is  generalized  for  J  input  parameters  and  concept  of  correction 
factors  based  on  analytical  ^nchmarks  is  introduced.  More  de¬ 
tailed  derivations  are  provided  by  Stem  et  al.  [13]. 

As  already  mentioned,  since  Stem  et  al.  [13]  there  has  been 
nearly  two  years  experience  with  present  approach,  especially 
through  nre  community  and  Gothenburg  2000  Workshop  on 
CFD  in  Ship  Hydrodynamics.  In  particular,  detailed  verification 
procedures  have  been  the  focus  of  attention  (Eca  and  Hoekstra 
[18];  Ebert  and  Gorski  [19]).  After  some  background  for  general¬ 
ized  RE  is  given,  two  approaches  for  estimating  errors  and  uncer¬ 
tainties  are  presented  and  are  based  on  (i)  correction  factors  pro¬ 
posed  in  the  current  paper  and  (ii)  factor  of  safety  approach 
proposed  by  Roache  (1998).  Finally,  a  discussion  of  fundamental 
and  practical  issues  for  verification  is  provided. 
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Background  for  Generalized  RE.  Generalized  RE  begins  with 
Eq.  (21).  The  error  terms  on  the  right-hand  side  of  Eq.  (21)  are  of 
known  form  (i.e.,  power  series  expansion  with  integer  powers  of 
Ax*)  based  on  analysis  of  the  modified  and  numerical  error  equa¬ 
tions  which  is  written  below  as  a  finite  sum  (i.e.,  error  estimate) 
and  for  the  kih  parameter  and  mth  solution 

R 

St  (Ax*  (27) 

«  1*1  " 

number  of  terms  retained  in  the  power  series,  powers  cor¬ 
respond  to  order  of  accuracy  (for  the  ith  term),  and  are  re¬ 
ferred  to  as  “grid”  functions  which  are  a  function  of  various  or¬ 
ders  and  combinations  of  derivatives  of  S  with  respect  to  .  It  is 
assumed  that  the  power  series  in  Eq.  (27)  is  convergent  (i.e.,  the 
finite  sum  convergence  to  the  infinite  series  value  as  more  terms 
are  included).  Substituting  Eq.  (27)  into  Eq.  (21)  results  in 

n  j 

2  Sf  (28) 

Subtraction  of  multiple  solutions  where  input  parameter  is 
uniformly  refined  eliminates  the  Sf^  terms  in  Eq.  (28)  since  Sf^  is 

independent  of  and  provides  equations  for  Sc ,  Pk^ »  and  ^ . 
This  assumes  ^  and  g^' ^  are  also  independent  of  Ajc/^  .  Since 
each  term  (i)  contains  2  unknowns,  m  =  2n+  1  solutions  are  re¬ 
quired  to  estimate  the  numerical  benchmark  Sc  and  the  first  n 
terms  in  the  expansion  in  Eq.  (28)  (i.e.,  for  «=  1,  m  =  3  and  for 
/i  =  2,  m-5,  etc).  The  accuracy  of  the  estimates  depends  on  how 
many  terms  are  retained  in  Eq.  (27),  the  magnitude  (importance) 
of  the  higher-order  terms,  and  the  validity  of  the  assumption  that 
and  g^  ^  are  independent  of  Ajc^t .  For  sufficiently  small  Ajc*  , 
the  solutions  are  in  the  asymptotic  range  such  that  higher-order 
terms  are  negligible  and  the  assumption  that  p[‘^  and  g*  ^  are  in¬ 
dependent  of  Ajcjt  is  valid.  However,  achieving  the  asymptotic 
range  for  practical  geometry  and  conditions  is  usually  not  possible 
and  m>3  is  undesirable  from  a  resources  point  of  view;  therefore, 
methods  are  needed  to  account  for  effects  of  higher-order  terms 
for  practical  application  of  RE.  Additionally,  methods  may  be 
needed  to  account  for  possible  dependence  of  and  g^*^  on 
Ajcjt ,  although  not  addressed  herein.  Usually  S*  is  estimated  for 
the  finest  value  of  the  input  parameter,  i.e.,  S*  =  corresponding 

to  the  finest  solution  Sl  , 

With  three  solutions  (m  =  3),  only  the  leading-order  term  of 
Eq.  (27)  can  be  estimated.  Solution  of  the  three  equations  for  Sc , 
and  g^'^  yields  estimates  for  the  error  and  order-of- 
accuracy  p* 


Pk= 


(29) 

ln(et3j/e*ji) 

In(rt) 

(30) 

medium  input  parameters  can  be  used  to  estimate  from 
Eq,  (29),  provided  that  Eq.  (30)  for  estimating  order  of  accuracy  is 
modified  as 


[ln(rj*-l)-ln(r^*-l)] 


(31) 


For  situations  when  '’jkji  (31)  is  a  transcendental  equa¬ 

tion  implicitly  defining  pjt  and  must  be  solved  iteratively.  If 
=  Eq.  (31)  degenerates  to  Eq.  (30). 

Estimating  Errors  and  Uncertainties  Using  Generalized  RE 
With  Correction  Factors.  Results  from  the  numerical  solution  of 
the  one-dimensional  (ID)  wave  and  two-dimensional  (2D) 
Laplace  equation  analytical  benchmarks  show  that  Eq.  (29)  has 
the  correct  form,  but  the  order  of  accuracy  is  poorly  estimated  by 
Eq.  (30)  except  in  the  asymptotic  range.  Analysis  of  the  results 
suggests  the  concept  of  correction  factors,  which  provide  a  quan¬ 
titative  metric  to  determine  proximity  of  the  solutions  to  the 
asymptotic  range,  account  for  the  effects  of  higher-order  terms, 
and  are  used  for  defining  and  estimating  errors  and  uncertainties. 
Details  are  provided  in  Appendix  A. 

Multiplication  of  Eq,  (29)  by  a  correction  factor  C*  provides  an 
estimate  for  accounting  for  the  effects  of  higher-order  terms 


(32) 


If  solutions  are  in  the  asymptotic  range,  correction  of  Eq.  (29)  is 
not  required  [i.e.,  Cjt=  1  so  ^at  Eqs.  (29)  and  (32)  are  equivalent]. 
For  solutions  outside  the  asymptotic  range,  Ct<  1  or  Ct>  1  indi¬ 
cates  that  the  leading-order  term  over  predicts  (higher-order  terms 
net  negative)  or  under  predicts  (higher-order  terms  net  positive) 
the  error,  respectively.  The  estimate  given  by  Eq.  (32)  includes 
both  sign  and  magnitude  and  is  used  to  estimate  U^  or  S*  and 
depending  on  how  close  the  solutions  are  to  the  asymptotic  range 
(i.e.,  how  close  is  to  1)  and  one’s  confidence  in  Eq.  (32).  There 
are  many  reasons  for  lack  of  confidence,  especially  for  complex 
three-dimensional  flows. 

For  Cjt  sufficiently  less  than  or  greater  than  1  and  lacking  con¬ 
fidence,  Uk  IS  estimated,  but  not  and  U^^.  Equation  (32)  is 

used  to  estimate  the  uncertainty  by  bounding  the  error  Sf  by  the 
sum  of  the  absolute  value  of  the  corrected  estimate  from  RE  and 
the  absolute  value  of  the  amount  of  the  correction 


1  +  1(1 -C*)5?£j  (33) 

*1  *1 

For  Ck  sufficiently  close  to  1  and  having  confidence,  Sf  and 
Uk^  are  estimated.  Equation  (32)  is  used  to  estimate  the  error  S*  , 
which  can  then  also  be  used  in  the  calculation  of  5^  [in  Eq.  (10)]. 
The  uncertainty  in  the  error  estimate  is  based  on  the  amount  of  the 
correction 


Solving  for  the  first-order  term  is  relatively  easy  since  evaluation 
of  Eqs.  (29)  and  (30)  only  requires  that  the  /n  =  3  solutions  are 
monotonically  convergent,  even  if  the  solutions  are  far  from  the 
asymptotic  range  and  Eqs.  (29)  and  (30)  are  inaccurate.  With  so¬ 
lutions  from  five  systematically  refined  input  parameters  (m 
=  5),  more  complicated  expressions  can  be  derived  to  estimate 
the  first  two  terms  of  the  power  series  expansion.  However,  their 
range  of  applicability  is  more  restrictive  since  all  five  solutions 
must  be  both  monotonically  convergent  and  sufficiently  close  to 
the  asymptotic  range  for  the  expressions  to  be  used. 

As  previously  mentioned,  solutions  from  three  values  of  input 
parameter  where  the  refinement  ratio  between  the  medium  and 
fine  input  parameters  is  not  equal  to  that  between  coarse  and 


(34) 

Note  that  in  the  limit  of  the  asymptotic  range,  =  1,  = 

=  ^^,andU,^=0. 

Two  definitions  for  the  correction  factor  were  developed.  The 
first  is  based  on  solution  of  Eq.  (32)  for  C*  with  based  on 

Eq.  (29)  but  replacing  pk  with  the  improved  estimate  pk 

(35) 

k 
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Similarly,  the  second  is  based  on  a  two-term  estimate  of  the  power 
series  which  is  used  to  estimate  where  pi^  and  are  re- 

placed  with  pi  and  qi 

^  est 


C*= 


(36) 


Pi^  and  are  estimates  for  limiting  orders  of  accuracy  of  the 
first  and  second  terms  of  the  error  expansion  equation  (27)  as 
spacing  size  goes  to  zero  and  the  asymptotic  range  is  reached, 
^nation  (35)  roughly  accounts  for  the  effects  of  higher-order 
terms  by  replacing  p*  with  thereby  providing  an  improved 
single-term  estimate.  Equation  (36)  more  rigorously  accounts  for 
higher-order  terms  since  it  is  derived  from  the  two-term  estimate 
with  first  and  second  term  order  of  accuracy  p[^^  and  p]^^  replaced 
by  and  Equation  (36)  simplifies  to  Eq.  (35)  in  the  limit 
of  the  asymptotic  range.  Both  correction  factors  only  require  so¬ 
lutions  for  three  parameter  values.  The  estimated  values  p^^^^  and 
Qv  can  be  based  either  on  the  assumed  theoretical  order  of  ac- 
curacy  p^^^  and  q^^^  or  solutions  for  simplified  geometry  and  con¬ 
ditions.  In  either  case,  preferably  including  the  effects  of  grid 
stretching. 

In  Appendix  A,  exact  (A)  and  numerical  (S)  solutions  are  used 
to  compare  the  true  simulation  error  (A  — 5)  to  (/)  an  uncorrected 
three-grid  error  estimate  using  Eq.  (29)  and  («)  corrected  esti¬ 
mates  based  on  Eq.  (32)  with  correction  factor  defined  by  Eq.  (35) 
or  (36).  Correction  of  error  estimates  with  both  definitions  of  C^^ 
results  in  improved  error  estimates.  Also,  uncertainty  estimates 
using  Eq.  (33)  with  correction  factor  defined  by  Eq.  (35)  or  (36) 
are  shown  to  bound  the  true  simulation  error  (A-S),  while  un¬ 
certainty  estimates  using  Eq.  (34)  are  shown  to  bound  the  differ¬ 
ence  between  the  corrected  solution  and  the  truth  (3^,-7),  Addi¬ 
tional  testing  of  expressions  for  C*  given  by  Eqs.  (35)  and  (36)  is 
needed  and  development  of  improved  expressions  within  the  pro¬ 
posed  general  framework  is  certainly  possible. 

Estimating  Uncertainties  Using  Generalized  RE  With  Factors 
of  Safety,  In  Roache  [3],  a  GCI  approach  is  proposed  where  a 
standard  three-grid  error  estimate  from  RE  is  multiplied  by  a  fac¬ 
tor  of  safety  F5  to  bound  the  simulation  error 


been  used  to  address  some  of  these  fundamental  issues  while  oth¬ 
ers  need  further  research.  Although  both  correction  factor  and 
factor  of  safety  approaches  were  presented,  the  authors  advocate 
the  use  of  former.  Results  from  the  numerical  solution  of  analytic 
benchmarks  show  that  the  factor  of  safety  approach  is  overly  con¬ 
servative,  especially  when  the  solutions  approach  the  asymptotic 
range  (Appendix  A).  This  is  in  contrast  to  the  variable  correction 
factor  approach  proposed  in  Eqs.  (33)  and  (34),  where  the  uncer¬ 
tainty  in  the  error  estimate  correctly  goes  to  zero  as  the  asymptotic 
range  is  approached  because  1 .  Admittedly,  others  have  rec¬ 
ommended  the  factor  of  safety  approach,  e.g.,  Eca  and  Hoekstra 
[18],  although  examination  of  their  results  as  with  our  own  analy¬ 
sis  indicates  that  such  estimates  are  overly  conservative. 

For  practical  applications,  especially  complex  flows  with  rela¬ 
tively  coarse  grids,  solutions  may  be  far  from  asymptotic  range 
such  that  some  variables  are  convergent  while  others  are  oscilla¬ 
tory  or  even  divergent.  Order  of  accuracy  and  therefore  correction 
factors  and  factors  of  safety  may  display  large  variability  indicat¬ 
ing  the  need  for  finer  grids.  Clearly,  more  than  3  grids  are  required 
to  estimate  errors  and  uncertainties  for  such  cases.  Eca  and  Hoek¬ 
stra  [18]  suggest  a  least-squares  approach  to  estimate  the  error  by 
computing  the  three  unknown  parameters  from  RE  when  more 
than  three  solutions  are  available.  The  behavior  of  the  asymptotic 
range  was  successfully  demonstrated  for  simpler  analytical  bench¬ 
marks  in  Appendix  A.  However,  the  existence  and  behavior  of  the 
asymptotic  range  for  practical  problems  has  not  been  demon¬ 
strated  due  to  lack  of  sufficiently  refined  grids,  number  of  solu¬ 
tions  to  assess  variability,  and  available  resources,  among  other 
issues.  Another  practical  issue  involves  selecting  and  maintaining 
appropriate  parameter  refinement  ratio  and  resources  for  obtaining 
solutions  with  sufficient  parameter  refinement  as  well  as  number 
of  solutions.  Lastly,  interpretation  of  results  is  an  issue  since,  as 
already  mentioned,  there  is  limited  experience  and  no  known 
solutions  for  practical  applications  in  the  asymptotic  range  for 
guidance. 

The  present  verification  procedures  represent  the  most  rational 
approach  presently  known.  However,  alternative  strategies  for  in¬ 
cluding  effects  of  higher-order  terms  may  be  just  as  viable,  e.g., 
treatment  of  the  power  series  exponents  as  known  integers  as 
proposed  by  Oberkampf  and  investigated  by  Eca  and  Hoekstra 
[18].  Once  available,  improved  verification  procedures  can  be  eas¬ 
ily  incorporated  into  the  present  overall  verification  and  validation 
methodology.  These  issues  are  discussed  further  in  Section  5  Con¬ 
clusions  and  Reconunendations  and  in  Part  2  (Wilson  et  al.  [12]). 


(37) 

*1 

Note  that  Eq.  (37)  with  factor  of  safety  differs  significantly  from 
Eq.  (34).  Herein  (e.r^.Pk.Pk^^^.qk^J.  in  contrast  to  Eq. 

(37)  where  is  a  constant  referred  to  as  a  factor  of  safety  F5 . 
The  exact  value  for  factor  of  safety  is  somewhat  ambiguous  and 
Roache  [3]  reconunends  1.25  for  careful  grid  studies  and  3  for 
cases  in  which  only  two  grids  are  used. 

Although  not  proposed  in  Roache  [3],  the  factor  of  safety  ap¬ 
proach  can  be  used  for  situations  where  the  solution  is  corrected 
with  an  error  estimate  from  RE.  Equation  (29)  is  used  to  estimate 
and  the  uncertainty  in  that  error  estimate  is  given  by 

Ut=iFs-i)KE,\  (38) 

Wth  this  approach,  a  fixed  percentage  of  a  three-grid  error 
estimate  (e.g.,  25%  ^^r  Fs- 1.25)  is  used  to  define  the  un- 

*1 

certainty  of  the  error  estimate  regardless  of  how  close  solutions 
are  to  the  asymptotic  range. 

Discussion  of  Fundamental  and  Practical  Issues,  Fundamen¬ 
tal  and  practice  issues  for  verification  are  discussed  in  this  sec¬ 
tion.  Fundamental  issues  include  convergence  of  power  series 
equation  (27),  assumptions  that  and  g*  ^  are  independent  of 
Axjt ,  and  estimating  pjt  Solution  of  analytical  benchmarks  has 


3.4  Oscillatory  Convergence.  For  oscillatory  convergence, 
i.e.,  condition  (11)  in  Eq.  (23),  uncertainties  can  be  estimated,  but 
not  the  signs  and  magnitudes  of  the  errors.  Uncertainties  are  esti¬ 
mated  based  on  determination  of  the  upper  (Sy)  and  lower  (Si) 
bounds  of  solution  oscillation,  which  requires  more  than  m  =  3 
solutions.  The  estimate  of  uncertainty  is  based  on  half  the  solution 
range 

Uk=\{Su-Si)  (39) 

3.5  Divergence.  For  divergence,  i.e.,  condition  (Hi)  in  Eq. 
(23),  errors  and  or  uncertainties  can  not  be  estimated.  The  prepa¬ 
ration  and  verification  steps  must  be  reconsidered.  Improvements 
in  iterative  convergence,  parameter  specification  (e.g.,  grid  qual¬ 
ity),  and/or  CFD  code  may  be  required  to  achieve  converging  or 
oscillatory  conditions. 

4  Validation  Procedures 

In  Section  2,  an  approach  for  assessing  the  simulation  modeling 
uncertainty  was  presented  where  for  successful  validation,  the 
comparison  error,  E  is  less  than  the  validation  uncertainty,  Uy 
given  by  Eqs.  (17)  and  (19)  for  uncorrected  and  corrected  solu¬ 
tions,  respectively.  In  this  section,  validation  procedures  are  pre¬ 
sented  through  discussions  in  Section  4.1  on  interpretation  of 
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validation  results  and  in  Section  4.2  on  use  of  corrected  simula¬ 
tion  results.  As  previously  mentioned,  Coleman  and  Stem  [6]  pro¬ 
vide  additional  discussion  on  validation  procedures. 

4.1  Interpretation  of  the  Results  of  a  Validation  Effort 
First,  consider  the  approach  in  which  the  simulation  numerical 
error  is  taken  to  be  stochastic  and  thus  the  uncertainty  Usn  is 
estimated.  From  a  general  perspective,  if  we  consider  the  three 
variables  C/y »  ^^reqd  six  combinations  (assuming 

none  of  the  three  variables  are  equal): 

1.  \E\<Uy<Ure,, 

2.  \E\<Ure,,<Uy 

3.  U,,,,<\E\<Uy 

4.  Uy<\E\<Ure,, 

5.  Uy<Ure,,<\E\ 

6.  Uregd<Uy<\E\  (40) 

In  cases  1,  2,  and  3,\E\<Uy\  validation  is  achieved  at  the  f/y 
level;  and  the  comparison  error  is  below  the  noise  level,  so  at¬ 
tempting  to  estimate  Ssma  is  not  feasible  from  an  uncertainty 
standpoint.  In  case  1,  validation  has  been  achieved  at  a  level 
below  Ureqd*  SO  Validation  is  successful  from  a  programmatic 
standpoint. 

In  cases  4, 5,  and  6,  Uy<\E\,  so  the  comparison  error  is  above 
the  noise  level  and  using  the  sign  and  magnitude  of  E  to  estimate 
SsMA  is  feasible  from  an  uncertainty  standpoint.  If  t/y<^|£|,  then 
E  corresponds  to  S^ma  ^ind  the  error  from  the  modeling  assump¬ 
tions  can  be  determined  unambiguously.  In  case  4,  validation  is 
successful  at  the  |£{  level  from  a  programmatic  standpoint. 

Now  consider  the  approach  in  which  the  simulation  numerical 
error  is  taken  to  be  deterministic  and  thus  and  the  uncertainty 
Uy^  are  estimated.  A  similar  set  of  comparisons  as  those  in  Eq. 
(40)  can  be  constructed  using  \Ec\r  Uy^,  and  Ureqd-  Since  Ec 
can  be  larger  or  smaller  than  £,  but  Uy^  should  always  be  less 
than  Uy ,  the  results  for  a  given  corrected  case  are  not  necessarily 
analogous  to  those  for  the  corresponding  uncorrected  case.  That 
is,  a  variable  can  be  validated  in  the  corrected  but  not  in  the 
uncorrected  case,  or  vice  versa.  For  cases  4,  5,  and  6  in  which 
Uy^<\Ec\,  one  can  argue  that  Ec  is  a  better  indicator  of  Ssma 
than  is  £,  assuming  that  one’s  confidence  in  using  the  estimate 
is  not  misplaced. 

4.2.  Use  of  Corrected  Versus  Uncorrected  Simulation  Re¬ 
sults.  As  previously  stated  in  Section  3.3,  the  requirements  for 
correcting  the  solution  are  that  the  correction  factor  be  close  to 
one  and  that  confidence  in  solutions  exist.  Since  the  variability  of 
the  order  of  accuracy  cannot  be  determined  from  solutions  on 
three  grids,  confidence  is  difficult  to  establish  in  this  case.  As  a 
result,  caution  should  be  exercised  when  correcting  solutions  us¬ 
ing  information  from  only  three  grids. 

If  a  validation  using  the  corrected  approach  is  successful  at  a 
set  condition,  then  if  one  chooses  to  associate  that  validation  un¬ 
certainty  level  with  the  simulation’s  prediction  at  a  neighboring 
condition  that  prediction  must  also  be  corrected.  That  means 
enough  runs  are  required  at  the  new  condition  to  allow  estimation 
of  the  numerical  errors  and  uncertainties.  If  this  is  not  done,  then 
the  comparison  error  £  and  validation  uncertainty  U  y  correspond¬ 
ing  to  the  use  of  the  uncorrected  S  and  its  associated  (larger)  Usn 
should  be  the  ones  considered  in  the  validation  with  which  one 
wants  to  associate  the  prediction  at  a  new  condition.  (Whether  to 
and  how  to  associate  an  uncertainty  level  at  a  validated  condition 
with  a  prediction  at  a  neighboring  condition  is  very  much  unre¬ 
solved  and  is  justifiably  the  subject  of  much  debate  at  this  time.) 


5  Conclusions 

The  present  comprehensive  approach  to  verification  and  valida¬ 
tion  methodology  and  procedures  sets  forth  concepts,  definitions, 
and  equations  derived  for  simulation  errors  and  uncertainties, 
which  provide  a  well-founded  mathematical  framework.  The  ap¬ 
proach  should  have  applicability  to  a  fairly  broad  range  of  CIT) 
codes,  including  RANS,  Navier-Stokes,  Euler,  boundary-element 
methods,  and  others.  However,  clearly  much  more  work  is  needed 
for  other  CFD  codes  (such  as  large-eddy  simulations),  additional 
error  sources,  and  alternative  error  and  uncertainty  estimation 
methods,  e.g.,  single-grid  methods  and  both  results  for  additional 
analytical  benchmarks  (especially  for  nonlinear  equations  and  us¬ 
ing  stretched  grids)  for  improved  definitions  of  correction  factors 
and  estimates  of  orders  of  accuracy,  and  alternative  strategies  to 
account  for  the  effects  of  higher-order  terms  in  RE.  Improved 
verification  procedures  once  available  can  be  easily  incorporated 
into  the  present  overall  verification  and  validation  methodology. 
Furthermore,  more  experience  is  needed  through  application  for 
different  codes  and  geometry  and  conditions,  especially  for  prac¬ 
tical  applications. 

As  mentioned  in  the  Introduction,  present  verification  and  vali¬ 
dation  methodology  and  procedures  were  recommended  and  used 
at  the  recent  Gothenburg  2000  Workshop  on  CFD  in  Ship  Hydro¬ 
dynamics  (Larsson  et  al.  [11]).  22  participating  research  groups 
from  12  countries  and  19  different  RANS  codes  were  used  for 
simulations  of  3  test  cases  representing  tanker,  container,  and  sur¬ 
face  combatant  hull  forms.  Most  groups  implemented  the  recom¬ 
mended  procedures,  but  lack  of  familiarity  with  the  procedures 
and  use  of  coarse  grids  led  to  difficulties.  Coarser  grid  solutions 
are  far  from  the  asymptotic  range  and  show  variability  such  that 
not  all  variables  display  monotonic  convergence  and  oscillatory 
convergence  and  even  divergence  is  evident.  For  monotonic  con¬ 
vergence,  variability  in  the  estimated  order  of  accuracy  was  ob¬ 
served  for  some  cases.  The  current  1  million  point  grids  are 
clearly  insufficient  for  more  complex  hull  forms  such  as  the  tanker 
and  an  order  of  magnitude  increase  in  points  may  be  required  to 
remove  variability  and  achieve  monotonic  convergence  for  most 
variables.  In  spite  of  difficulties,  the  effort  was  beneficial  in  en¬ 
abling  quantitative  evaluation  of  levels  of  verification  and  valida¬ 
tion,  increasing  familiarity  with  verification  and  validation  proce¬ 
dures,  interpretation  of  results,  and  identification  of  grid 
requirements  for  decreasing  levels  of  errors  and  uncertainties. 
Careful  examination  of  verification  results  even  for  relatively 
coarse  grid  solutions  provides  a  road  map  towards  achieving  ac¬ 
ceptable  levels  of  verification. 

Verification  and  validation  methodology  and  procedures  should 
be  helpful  in  guiding  future  developments  in  CFD  through  docu¬ 
mentation,  verification,  and  validation  studies  and  in  transition  of 
CFD  codes  to  design  through  establishment  of  credibility.  Pre¬ 
sumably,  with  a  sufficient  number  of  documented,  verified,  and 
validated  solutions  along  with  selected  verification  studies,  a  CFD 
code  can  be  accredited  for  a  certain  range  of  applications.  The 
contribution  of  the  present  work  is  in  providing  methodology  and 
procedures  for  the  former,  which  hopefully  will  help  lead  to  the 
latter. 

Part  2  provides  an  example  for  RANS  simulations  for  a  cargo/ 
container  ship  where  issues  with  regard  to  practical  application  of 
the  methodology  and  procedures  and  interpretation  of  verification 
and  validation  results  are  discussed  (Wilson  et  al.  [12]). 
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Nomenclature 

Ck  =  correction  factor 
D  -  benchmark  data 
EyEc  =  comparison  error,  corrected 
Pi  =  order  of  accuracy 
Rjt  =  parameter  refinement  ratio 
S,Sc  =  simulation  result,  corrected 
T  =  truth 

U  =  uncertainty  estimate 
Ud  =  data  uncertainty 

=  comparison  error  uncertainty,  corrected 
17/  =  iteration  uncertainty 

Vp^V Pc  —  parameter  uncertainty  (e.g.,  grid  size  G  and 
time  step  7),  corrected 

^reqd  “  programmatic  validation  requirement 
=  simulation  uncertainty,  corrected 
UsM  ^  simulation  modeling  uncertainty 
UsMA  =  simulation  modeling  assumption  uncertainty 
UspD  ~  simulation  uncertainty  due  to  use  of  previous 
data 

^STEf^ScTE  =  simulation  total  estimated  numerical  uncertainty 
—  simulation  numerical  uncertainty,  corrected 

Uy^Uvc  =  validation  uncertainty,  corrected 

Ajfjt  =  increment  in  kth  input  parameter  (e.g.,  grid  size 
G  and  time  step  70 
S  =  error 

S*  =  error  estimate  with  sign  and  magnitude 
Sj.Sf  =  iteration  error,  estimate 
Sp^Sp  =  parameter  error,  estimate 
”  simulation  error,  corrected 
SsN  =  simulation  numerical  error 
Ssma  ~  simulation  modeling  assumption  error 
e  =  solution  change 
Bsn  =  error  in  S* 

Appendix  A.  Analytical  Benchmarks 

The  use  of  analytical  benchmarks  for  development  of  the  con¬ 
cept  of  correction  factors  as  discussed  in  Section  3.3  is  presented 
in  this  Appendix.  For  analytical  benchmarks,  the  modeling  error  is 
zero  such  that  the  simulation  error  is  solely  due  to  numerical  error. 
Results  are  obtained  for  two  analytical  benchmarks  one¬ 
dimensional  (ID)  wave  and  two-dimensional  (2D)  Laplace  ^na¬ 
tions.  The  results  for  the  2D  Laplace  equation  were  qualitatively 
similar  to  those  for  the  ID  wave  equation,  which  are  presented. 
Exact  solutions  from  analytical  benchmarks  are  used  to  determine 
the  exact  simulation  numerical  error  which  is  compared  to  esti¬ 
mates  from  RE,  including  use  of  correction  factors.  More  details 
are  provided  in  Stem  etal.  [13],  including  single  grid  error 
estimates. 

Verification  of  Analytical  Benchmarks.  For  verification  us¬ 
ing  an  analytical  benchmark,  the  simulation  error  and  uncertainty 
are  given  by  Ss^S-A^Ssn  and  while  the  corrected 

simulation  error  and  uncertainty  are  given  by  Ssc=Sc^A  =  esN 
and  Ulc-^l^N^  Simulations  are  verified  if  l£|  =  |A-Sl<U5;^ 
and  corrected  simulations  are  verified  if  \Ec\  =  jA  —  Sd <  t/ s^n  • 

The  first-order,  linear  ID  wave  equation  models  the  behavior  of 
a  more  complicated  (nonlinear)  partial  differential  equation.  The 
initial  condition  is  prescribed  by  a  Gaussian  function  centered  at 
jc=0.0.  Two  discretization  techniques  were  studied:  (/)  first-order 
(Euler)  explicit  method  with  first-order  upwind  spatial  discretiza- 


Fig.  2  Verification  results  for  first-order  numerical  solution  of 
1D  wave  equation,  (a)  Comparison  of  true  error  A-SXo  esti¬ 
mates  from  RE,  (b)  correction  factor,  and  (c)  comparison  of 
|A-Scl  and  Uscnj  and  (d)  comparison  of  [A- SI  and  Usw 


tion;  and  (ii)  a  second-order  implicit  method  with  second-order 
central  spatial  discretization.  Since  trends  from  both  schemes  are 
similar,  only  the  results  from  the  first-order  scheme  are  presented. 

A  combined  grid  size  and  time  step  study  was  performed  where 
ten  solutions  were  obtained  by  successively  doubling  both  the  grid 
and  time  step  such  that  Af/Ax=0.5  for  all  solutions.  With  this 
approach,  solutions  changes  are  used  to  estimate  total  (temporal 
and  spatial)  simulation  errors  and  uncertainties.  Accordingly,  the 
generic  subscript  appearing  in  expressions  for  errors  and  un¬ 
certainties  in  Section  3.3  is  replaced  with  ‘SAT  in  this  section 
where  appropriate. 

Errors,  Uncertainties,  and  Correction  Factors.  The  con¬ 
cept  of  a  multiplication  correction  factor  was  introduced  in  Sec¬ 
tion  3.3.  The  correction  factor  C;.  was  used  to  define  the  numerical 
uncertainty  in  Eq.  (33)  or  when  conditions  permit  to  improve  error 
estimates  in  Eq.  (32)  and  to  define  the  uncertainty  in  that  error 
estimate  in  Eq.  (34).  Error  and  uncertainty  estimates  given  by  Eqs. 
(32) -(34)  are  tested  by  numerical  solution  of  analytical  bench¬ 
marks  as  well  as  development  of  expressions  for  correction  factor. 

Figure  2(a)  compares  the  true  simulation  error  E  to  the  three- 
grid  error  estimate  3%^^  front  Eq.  (29)  versus  step  size  at  one 
spatial  location  (jc=1  since  maximums  of  numerical  error  occur 
there).  The  three-grid  estimate  accurately  estimates  the  true  error 
E  for  smaller  step  sizes,  but  over  predicts  E  for  larger  step  sizes. 
Closer  examination  reveals  that  Eq.  (29)  over  estimates  the  error 
because  Eq.  (30)  under  estimates  the  order  of  accuracy,  as  also 
shown  in  Fig.  2(a). 

Two  definitions  for  were  investigated.  The  first  is  based  on 

solving  equation  (32)  for  with  Spp  defined  in  Eq.  (29)  but 

*1 

replacing  p*  with  the  improved  estimate  which  is  provided 
by  Eq.  (35)  where  Pk  is  an  estimate  of  the  limiting  order  of 
accuracy  of  the  first  term  of  the  error  expansion  equation  (27). 
Similarly,  the  second  definition  of  correction  factor  is  based  on 
estimating  Spp  using  the  first  two  terms  of  the  powers  series  and 

replacing  Pk  and  qk  with  improved  estimates  Pk^^^  and  which 
is  provided  by  Eq.  (36)  where  Pk^^^  estimates  for 

limiting  orders  of  accuracy  of  the  first  and  second  terms  of  the 
error  expansion  equation  (27)  as  spacing  size  goes  to  zero  and  the 
asymptotic  range  is  reached.  With  this  definition,  correction  fac¬ 
tors  approach  one  in  the  limit  of  zero  spacing  size.  The  estimated 
values  Pk  and  qk  can  be  based  either  on  the  assumed  theoret- 
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ical  order  of  accuracy  and  or  solutions  for  simplified 
geometry  and  conditions.  In  either  case,  preferably  including  the 
effects  of  grid  stretching. 

Figure  2(a)  also  compares  the  true  error  E  to  (i)  an  uncorrected 
three-grid  error  estimate  using  Eq.  (29)  and  (zi)  corrected  esti¬ 
mates  based  on  Eq.  (32)  with  correction  factor  defined  by  Eq.  (35) 
or  (36).  Both  estimates  are  closer  to  E  than  the  uncorrected  three 
grid  estimate  ,  but  for  coarser  grids  is  somewhat  too 
small  and  is  slightly  too  large.  Figure  2{b)  shows  the  same 
trends,  but  directly  compares  the  exact  correction  factor  to 

Eqs.  (35)  and  (36).  In  this  case,  Cjt<  1  indicates  that  the  leading- 
order  term  over  predicts  (higher-order  terms  net  negative)  the  er¬ 
ror.  However,  for  the  general  case,  is  equally  likely  to  be  <1  or 
>1  depending  whether  the  order  of  accuracy  is  approached  from 
below  or  above,  respectively.  C*>  1  indicates  that  the  leading- 
order  term  under  predicts  (higher-order  terms  net  positive)  the 
error.  Thus,  for  the  general  case  the  correction  to  the  leading-terra 
error  estimate  is  equally  likely  to  be  positive  or  negative  and  can 
be  used  to  define  the  simulation  numerical  uncertainty. 

For  Cjfc  sufficiently  close  to  1  and  having  confidence,  Sf  and 
are  estimated.  Correction  factors  and  are  used  to 

estimate  the  error  Sf  in  Eq.  (32)  which  can  then  also  be  used  in 
the  calculation  of  Sc  [in  Eq.  (10)]  and  uncertainty  in  Eq.  (34). 
Figure  2(c)  shows  a  comparison  of  |i4  “-5^1  and  three  uncertainty 
estimates:  (0  defined  using  (iz)  defined  using 
;  and  (zzz)  Us^n  ^  f&ctor  of  safety  approach  given  by  Eq. 
(38)  with  Fs- 1.25.  The  results  show  that  the  uncertainty  estimate 
successfully  bounds  |A-5cl  over  the  entire  range  of  step 
sizes  and  that  is  not  conservative  enough  since  Uf^<\A 
-5c|.  The  uncertainty  estimate  based  on  the  factor  of  safety  ap¬ 
proach  is  not  conservative  enough  for  the  coarsest  two  grids  and  is 
overly  conservative  by  an  order  of  magnitude  for  the  four  finest 
grids  (i.e.,  when  solutions  are  in  the  asymptotic  range).  For  C* 
sufficiently  less  than  or  greater  than  1  and  lacking  confidence,  Uk 
is  estimated,  but  not  S*  and  Correction  factors  and 
are  used  to  estimate  the  uncertainty  in  Eq.  (33)  which  is  compared 
to  factor  of  safety  approach  given  by  Eq.  (37).  Figure  2{d)  shows 
that  all  three  uncertainty  estimates  successfully  bound  the  true 
error  |A“Sl  although  the  factor  of  safety  approach  is  overly  con¬ 
servative  for  all  grids. 

Uncertainty  estimates  enable  a  quantitative  measure  of  verifica¬ 
tion  for  analytical  benchmarks.  Figure  2(c,  d)  indicates  that  the 
present  solutions  are  verified  over  the  chosen  range  of  grid  size 
and  time  step.  As  expected,  the  largest  levels  of  uncertainty  are  for 
the  coarsest  grid  size  and  time  step  where  levels  are  (f/jt,l/tp 
=  (15%,7.5%). 

Eca  and  Hoekstra  [18]  also  perform  verification  for  the  2D 


Laplace  equation  analytical  benchmark.  Their  results  are  consis¬ 
tent  with  our  own  in  showing  that  uncertainty  estimates  using  Eq, 
(33)  always  bounded  the  true  error.  Unlike  our  own  results,  their 
results  indicate  that  the  uncertainty  estimate  from  Eq.  (34)  failed 
to  bound  the  difference  in  the  truth  and  numerical  benchmark  for 
some  grid  triplets  when  the  apparent  order  of  accuracy  was  esti¬ 
mated  to  be  larger  than  the  theoretical  value. 
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Comprehensive  Approach  to 
Verification  and  Validation  of  CFD 
Simulations— Part  2:  Application 
for  Bans  Simulation  of  a 
Cargo/Container  Ship 

Part  2  of  this  two-part  paper  provides  an  example  case  study  following  the  recently 
developed  comprehensive  verification  and  validation  approach  presented  in  Part  I.  The 
case  study  is  for  a  RANS  simulation  of  an  established  benchmark  for  ship  hydrodynamics 
using  a  ship  hydrodynamics  CFD  code.  Verification  of  the  resistance  (integral  variable) 
and  wave  profile  (point  variable)  indicates  iterative  uncertainties  much  less  than  grid 
uncertainties  and  simulation  numerical  uncertainties  of  about  2%Sj  (Sj  is  the  simulation 
value  for  the  finest  grid).  Validation  of  the  resistance  and  wave  profile  shows  modeling 
errors  of  about  8%D  (D  is  the  measured  resistance)  and  6%^^^  is  the  maximum 
wave  elevation),  which  should  be  addressed  for  possible  validation  at  the  3%D  and 
levels.  Reducing  the  level  of  validation  primarily  requires  reduction  in  experi¬ 
mental  uncertainties.  The  reduction  of  both  modeling  errors  and  experimental  uncertain¬ 
ties  will  produce  verified  and  validated  solutions  at  low  levels  for  this  application  using 
the  present  CFD  code.  Although  there  are  many  issues  for  practical  applications,  the 
methodology  and  procedures  are  shown  to  be  successful  for  assessing  levels  of  verifica¬ 
tion  and  validation  and  identifying  modeling  errors  in  some  cases.  For  practical  appli¬ 
cations,  solutions  are  far  from  the  asymptotic  range:  therefore,  analysis  and  interpretation 
of  the  results  are  shown  to  be  important  in  assessing  variability  for  order  of  accuracy, 
levels  of  verification,  and  strategies  for  reducing  numerical  and  modeling  errors  and 
uncertainties,  [DOI:  10.1115/1.1412236] 


1  Introduction 

Computational  fluid  dynamics  (CPI!))  is  fast  becoming  an  inte¬ 
gral  tool  in  the  engineering  design  process  as  it  is  applied  to 
increasing  complex  geometry  and  physics.  As  with  the  use  of 
experimental  fluid  dynamics  (EFD)  in  making  design  decisions, 
assessment  of  quality  of  results  is  imperative,  which  has  acceler¬ 
ated  progress  on  development  of  verification  and  validation 
(V&V)  methodology  and  procedures  for  estimating  numerical  and 
modeling  errors  and  uncertainties  in  CFD  simulations.  However, 
in  spite  of  the  progress,  the  various  viewpoints  have  not  yet  fully 
converged  and  current  methodology  and  procedures  are  not  yet 
standardized.  Case  studies  are  important  for  evaluating  various 
current  V&V  approaches  and  achieving  standardization.  Mehta 
[1]  provides  several  case  studies  following  different  approaches 
for  a  variety  of  applications.  In  some  cases,  large  differences  in 
approaches,  use  of  different  test  cases,  and  incomplete  documen¬ 
tation  makes  evaluation  difficult. 

Part  2  of  the  present  two-part  paper  provides  an  example  case 
study  following  the  recently  developed  comprehensive  V&V  ap¬ 
proach  presented  in  Part  1  (Stem  et  al.  [2]).  The  case  study  is  for 
a  RANS  simulation  of  an  established  benchmark  for  ship  hydro¬ 
dynamics  using  a  current  ship  hydrodynamics  CFD  code.  How¬ 
ever,  the  V&V  approach  is  equally  applicable  to  CFD  simulations 
for  other  applications  in  fluids  engineering  such  as  aerospace,  en¬ 
vironmental,  and  automotive  and  also  should  be  applicable  to  a 
fairly  broad  range  of  codes  such  as  boundary  element  methods 
and  certain  aspects  of  large-eddy  and  direct  numerical  simula- 


Contributed  by  the  Fluids  Engineering  Division  for  publication  in  the  JOURNAL 
OF  FLUIDS  Engineering.  Manuscript  received  by  the  Fluids  Engineering  Division 
November  4,  1999;  revised  manuscript  received  July  10,  2001.  Associate  Editor 
P.  E.  Raad. 


tions.  The  present  papers  are  based  on  Stem  et  al.  [3],  but  with 
improved  presentation  and  discussion  based  on  nearly  two  years 
experience  with  the  present  V&V  approach,  especially  through  the 
International  Towing  Tank  Conference  (ITTC)  community  and 
Gothenburg  2000  Workshop  on  CFD  in  Ship  Hydrodynamics 
(Larsson  et  al.). 

The  specific  objectives  of  the  present  work  are:  (/)  to  provide  a 
documented  solution  following  the  methodology  and  procedures 
in  Part  1;  (ii)  to  address  practical  issues  in  V&V  of  CFD  simula¬ 
tions  for  complex  geometries  (e.g.,  generation  of  systematic 
grids);  and  (i7i)  to  provide  analysis  and  discussion  of  V&V  results 
for  a  practical  application  where  interpretation  is  complicated  due 
to  variability  in  the  order  of  accuracy.  Verification  and  validation 
methodology  and  procedures  are  suimnarized  in  Section  2  fol¬ 
lowed  by  a  description  of  the  CFD  code  in  Section  3.  Next,  ge¬ 
ometry,  conditions,  and  benchmark  data  are  specified  in  Section  4 
with  issues  related  to  grid  studies  for  practical  applications  given 
in  Section  5.  V&V  results  for  the  total  resistance  and  wave  profile 
are  presented  in  Sections  6  and  7,  respectively.  Finally,  conclu¬ 
sions  are  given  in  Section  8. 


2  Verification  and  Validation  Methodology  and 
Procedures 

The  V&V  methodology  and  procedures  set  forth  in  Part  1  pro¬ 
vide  a  pragmatic  approach  for  estimating  simulation  errors  and 
uncertainties.  The  philosophy  is  strongly  influenced  by  EFD  un¬ 
certainty  analysis.  The  present  approach  allows  for  treatment  of 
simulation  errors  as  either  stochastic  or  deterministic  and  properly 
takes  into  account  uncertainties  in  both  the  simulation  and  the  data 
in  assessing  the  level  of  validation. 
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Methodology.  The  simulation  error  Ss  is  defined  as  the  dif¬ 
ference  between  a  simulation  result  S  and  the  truth  T  and  is  com¬ 
posed  of  modeling  Ssm  and  numerical  Ssn  errors  {Ss=S-T 
=  Ssm-^Ssn)  with  the  corresponding  simulation  uncertainty  given 
by  U]=Ulnf+U]i^.  For  certain  conditions,  both  the  sign  and 
magnitude  of  the  numerical  error  can  be  estimated  as  Sss^  ^sn 
+  ess  where  is  an  estimate  of  the  sign  and  magnitude  of  Ssn 

and  Ssn  is  the  error  in  that  estimate.  The  simulation  value  is 
corrected  to  provide  a  numerical  benchmark  Sc,  which  is  defined 

by 

5^=5— (1) 

with  error  equation  Ss^-Sc-T-  Ssm'^ ^sn  and  corresponding 
uncertainty  equation  where  Us^  is  the  uncer¬ 

tainty  in  the  corrected  simulation  and  Us^n  the  uncertainty 
estimate  for  bsn  • 

Verification  is  defined  as  a  process  for  assessing  simulation  nu¬ 
merical  uncertainty  Usn  and,  when  conditions  permit,  estimating 
the  sign  and  magnitude  of  the  simulation  numerical  error 
itself  and  the  uncertainty  in  that  error  estimate  Numerical 

error  is  decomposed  into  contributions  from  iteration  number  , 
grid  size  Sc ,  time  step  Sj,  and  other  parameters  Sp,  which  gives 
the  following  expression  for  the  simulation  numerical  uncertainty 

ul^=uj+ul+ul+ul  (2) 

For  situations  when  the  solution  is  corrected  to  produce  a  numeri¬ 
cal  benchmark  Sc ,  the  estimated  simulation  numerical  error 
and  corrected  uncertainty  Us^n  by 


ducted.  Changes  between  medium-fine  coarse- 

medium  Bt  -Sk  solutions  are  used  to  define  the  conver- 

*32  *^3  *2 

gence  ratio 

(9) 

K  K21  ^32 

and  to  determine  convergence  condition  where  5*^,  Sk^  cor¬ 
respond  to  solutions  with  fine,  medium,  and  coarse  input  param¬ 
eter,  respectively,  corrected  for  iterative  errors.  Three  convergence 
conditions  are  possible: 

(i)  Monotonic  convergence:  0<Rk<l 

{ii)  Oscillatory  convergence:  /?*<0  (10) 

(ill)  Divergence:  /?*<! 

For  condition  (i),  generalized  RE  is  used  to  estimate  U*  or  <5*  and 
Uk  .  For  condition  (ii),  uncertainties  are  estimated  simply  by  at¬ 
tempting  to  bound  the  error  based  on  oscillation  maximums  Sy 
and  minimums  5^,  i.e.,  f/jt=  1/2(5 jy—St).  For  condition  (Hi), 
errors  and  uncertainties  cannot  be  estimated. 

For  convergence  condition  (i),  generalized  RE  is  used  to  esti¬ 
mate  the  error  due  to  selection  of  the  M  input  parameter 

and  order-of-accuracy  Pk 


S*  =  — 

ln(Cfc32/e*2,) 

In(rt) 


(11) 

(12) 


Sfn=St+S^+S^  +  S^  (3) 

U]^N=Ul+Ul^+Ul^+Ul^  (4) 

Validation  is  defined  as  a  process  for  assessing  simulation  mod¬ 
eling  uncertainty  Usm  by  using  benchmark  experimental  data  and, 
when  conditions  permit,  estimating  the  sign  and  magnitude  of  the 
modeling  error  Ssm  ilself.  The  comparison  error  E  is  given  by  the 
difference  in  the  data  D  and  simulation  S  values 

E=D^ S— Sp-^iSsMA"^  ^sn) 

where  Ssm  bas  been  decomposed  into  the  sum  of  Sspp,  error 
from  the  use  of  previous  data  such  as  fluid  properties,  and  Ssma  * 
error  from  modeling  assumptions.  To  determine  if  validation  has 
been  achieved,  E  is  compared  to  the  validation  uncertainty  U  y 
given  by 

ul=ul+ul^+uli,^  (6) 


Correction  of  Eq.  (11)  through  a  multiplication  factor  C*  accounts 
for  effects  of  higher-order  terms  and  provides  a  quantitative  met¬ 
ric  to  determine  proximity  of  the  solutions  to  the  asymptotic  range 


Sl-CkS^E.^Ck 
1  *1 


c*. 


rj*-l 


(13) 


where  the  correction  factor  is  given  by 


Ck= 


(14) 


and  Pk  is  an  estimate  for  the  limiting  order  of  accuracy  as  spac¬ 
ing  size  goes  to  zero  and  the  asymptotic  range  is  reached  so  that 
C|t— ►!.  When  solutions  are  far  from  the  asymptotic  range.  Ck  is 
sufficiently  less  than  or  greater  than  1  and  only  the  magnitude  of 
the  error  is  estimated  through  the  uncertainty  Uk 


(/t=lQ5j£j  +  l(l-Q)5j£j  (15) 

*1  *1 


If  \E\<Uy,  the  combination  of  all  the  errors  in  D  and  5  is  smaller 
than  Uy  and  validation  is  achieved  at  the  Uy  level.  If  Uy<\E\j 
the  sign  and  magnitude  of  E  =  ^sma  be  used  to  make  model¬ 
ing  improvements.  For  the  corrected  approach,  the  equations 
equivalent  to  Eqs.  (5)  and  (6)  are 

Ec—D  —  Sc^SD~i^SMA‘^^SPD'^^SN) 

Procedures.  The  overall  CFD  V&V  procedures  cm  be  con¬ 
veniently  grouped  into  four  consecutive  steps:  preparation,  verifi¬ 
cation,  validation,  and  documentation. 

Verification  is  accomplished  through  parameter  converpnce 
studies  using  multiple  solutions  (at  least  3)  with  systematic  pa¬ 
rameter  refinement  by  varying  the  kth  input  parameter  Axjt  while 
holding  all  other  parameters  constant.  Iterative  errors  must  be  ac¬ 
curately  estimated  or  negligible  in  comparison  to  errors  due  to 
input  parameters  before  accurate  convergence  studies  can  be  con- 


When  solutions  are  close  to  the  asymptotic  range,  C*  is  close  to  1 
so  that  Sk  is  estimated  using  Eq.  (13)  and  Uk^  is  estimated  by 

Alternatively,  a  factor  of  safety  approach  proposed  in  Roache 
(1998)  can  be  used  to  define  Uk  and  Uk^- 

Validation  is  accomplished  through  comparison  of  simulations 
with  benchmark  EFD  data,  including  experimental  uncertainty  es¬ 
timates  Up,  If  the  three  variables  Uy,  |Ei,  and  U^egd  (program¬ 
matic  validation  requirement)  are  considered,  there  are  six  com¬ 
binations.  For  three  cases,  |E|<l/v  validation  is  achieved  at 
the  U V  level,  but  for  only  one  of  these  Uy<U ^eqd  so  that  valida¬ 
tion  is  also  achieved  at  Ureqd-  In  these  cases,  attempting  to  esti¬ 
mate  modeling  errors  Ssma  not  feasible  from  an  uncertainty 
standpoint.  For  the  three  other  cases,  Uy<\E\  and  using  the  sign 
and  magnitude  of  E  to  estimate  Ssma  ‘s  feasible  from  an  uncer¬ 
tainty  standpoint.  In  one  of  these  cases,  Uy<\E\<Ureqd  so  that 
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validation  is  successful  at  the  \E\  level  from  a  programmatic  stand¬ 
point.  Similar  conclusions  can  be  reached  using  the  corrected 
comparison  error  and  corrected  validation  uncertainty. 

3  RANSCFDCode 

Solutions  are  obtained  with  CFDSHIP-IOWA,  which  is  a 
general-purpose,  multi-block,  MPI-based  high-performance,  un¬ 
steady  RANS  CFD  code  (Paterson  and  Sinkovits,  [4];  Paterson 
et  al.  [5];  Wilson  et  al.  [6])  developed  for  computational  ship  hy¬ 
drodynamics.  The  three-dimensional  unsteady  incompressible 
RANS  equations  are  solved  in  either  Cartesian  or  cylindrical-polar 
base  coordinate  systems.  The  grid  dynamically  conforms  to  the 
solution  of  the  exact  kinematic  ffee-surface  boundary  condition. 
Approximate  dynamic  ffee-surface  boundary  conditions  provide 
boundary  conditions  for  velocity  and  pressure.  For  production 
simulations,  Reynolds-stress  closure  is  accomplished  using  the 
standard  linear  stress-strain  relationship  and  a  variety  of  eddy- 
viscosity  models  including  algebraic  Baldwin-Lomax  and  two- 
equation  k-o)  and  k-e  models.  The  solution  scheme  is  based  upon 
the  PISO  algorithm  and  is  fully  implicit.  The  convective  and  vis¬ 
cous  terms  are  discretized  with  second-order  upwind  and  second- 
order  central  differences,  respectively.  Although  the  code  can  be 
run  in  either  steady  state  or  time-accurate  mode,  a  time-marching 
procedure  was  used  to  obtain  steady-state  RANS  solutions  for 
simulations  in  this  paper.  The  pressure  equation  is  obtained  by 
taking  the  divergence  of  the  momentum  equations.  Further  de¬ 
scription  can  be  found  at  http:/AvwwMhrMiowa.edu/--cfdship 
where  related  references  may  be  found  in  electronic  form. 

4  Geometry,  Conditions,  and  Benchmark  Data 

Steady-state  simulations  are  performed  for  the  Series  60  cargo/ 
container  ship.  The  Series  60  was  used  for  two  of  the  three  test 
cases  at  an  international  workshop  on  validation  of  ship  hydrody¬ 
namics  CFD  codes  (CFD  Workshop  Tokyo  [7]).  The  conditions 
for  the  calculations  are  Froude  number  Fr=0.316,  Reynolds 
number  /?e  =  4.3X  10^,  and  model  orientation  with  zero  sinkage 
and  trim.  These  are  the  same  conditions  as  the  experiments,  ex¬ 
cept  the  resistance  and  sinkage  and  trim  tests,  as  explained  next. 
The  variables  selected  for  verification  and  validation  are  resis¬ 
tance  Ct  (integral  variable)  and  wave  profile  (  (point  variable). 

The  benchmark  data  is  provided  by  Toda  et  al.  [8],  which  was 
also  the  data  used  for  the  Series  60  test  cases  at  the  CFD  Work¬ 
shop  Tokyo  [7].  The  data  include  resistance  and  sinkage  and  trim 
for  a  range  of  Fr  for  the  model  free  condition  (i.e.,  free  to  sink  and 
trim).  Also,  wave  profiles,  near-field  wave  pattern,  mean  veloci¬ 
ties,  and  pressures  at  numerous  stations  from  the  bow  to  the  stem 
and  near  wake,  all  for  Fr=  (0.16,0.316)  and  the  zero  sinkage  and 
trim  model  fixed  condition.  The  data  also  include  uncertainty  es¬ 
timates,  which  were  recently  confirmed/updated  by  Longo  and 
Stem  [9]  closely  following  standard  procedures  (Coleman  and 
Steele  [10]). 

The  resistance  is  known  to  be  larger  for  free  versus  fixed  mod¬ 
els.  Data  for  the  Series  60  indicates  about  an  8%  increase  in  C^.  for 
the  free  versus  fixed  condition  over  a  range  of  Fr  including  Fr 
=  0.316  (Ogiwara  and  Kajatani  [11]).  The  Toda  et  al.  [8]  resis¬ 
tance  values  were  calibrated  (i.e.,  reduced  by  8%)  for  effects  of 
sinkage  and  trim  for  the  present  comparisons. 

5  Grids  Studies  With  Systematic  Refinement 

Errors  and  uncertainties  due  to  grid  size  are  estimated  using 
multiple  solutions  (at  least  3)  on  systematically  refined  grids  with 
constant  refinement  ratio,  Although 

grid  doubling  (rc=2)  is  typically  used  for  simplicity,  resolving 
similar  physics  and  meeting  near-wall  spacing  requirements  for 
turbulence  modeling  can  be  prohibitively  difficult,  especially  in 
3D  simulations.  A  more  reasonable  refinement  ratio  of  rG=v5 
was  recommended  in  Part  1  and  is  used  here.  Using  r^-Vi  has 
the  added  benefit  that  the  coarse  grid  can  be  easily  generated  by 
removing  every  other  fine  grid  point. 


Table  1  Grid  dimensions  and  y+  values  for  grid  refinement 
studies 


Grid 

Grid 

Dimensions 

Total  Number 

of  points 

y+ 

1 

287x78x43 

876,211 

0.7 

2 

201x51x31 

317,781 

1 

3 

144x36x22 

114,048 

.  lA 

4 

101x26x16 

42,016 

2 

Grid  studies  were  conducted  using  four  grids,  which  enables 
two  separate  three-grid  studies  to  be  performed  and  compared. 
Grid  study  1  (GSl)  gives  estimates  for  grid  errors  and  uncertain¬ 
ties  on  grid  1  using  the  three  finest  grids  1-3  while  grid  study  2 
(GS2)  gives  estimates  for  grid  errors  and  uncertainties  on  grid  2 
using  the  three  coarsest  grids  2-4.  Grid  dimensions  and  average 
y'^  values  (for  grid  points  closest  to  no-slip  surface)  are  given  in 
Table  1  and  a  comparison  of  the  four  grids  at  the  free  surface 
plane  is  shown  in  Fig.  1  along  with  computed  wave  elevation 
contours. 

With  grid  refinement  ratio  rc=v5,  only  grids  1  and  2  were 
generated  manually  using  the  commercial  code  GRIDGEN  (Point- 
wise,  Inc.).  Grids  3  and  4  were  obtained  by  removing  every  other 
point  from  grids  1  and  2,  respectively  (i.e.,  the  grid  spacing  of 
grids  3  and  4  is  twice  that  of  grids  1  and  2,  respectively).  In  an 
effort  to  keep  the  exact  shape  of  the  leading  and  trailing  edges  of 
the  hull  surface  on  all  four  grids,  the  single-block  grid  system  is 
divided  into  three  subblocks.  The  7=1  surface  of  one  of  the  sub¬ 
blocks  is  body-fitted  and  defines  the  entire  no-slip  surface  of  the 


Fig.  1  Grids  and  computed  wave  contours  on  the  free-surface 
plane  from  verification  and  validation  studies  for  Series  60:  (a) 
and  (b)  coarsest— grid  4;  (c)  and  (d)  grid  3;  (e)  and  (f )  grid  2; 
and  ig)  and  (h)  finest— grid  1.  Ship  leading  and  trailing  edges  at 
x/L=0  and  1,  respectively. 
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ship  from  the  leading  to  trailing  edges.  The  1  surface  of  the 
other  two  subblocks,  one  upstream  and  one  downstream  of 
the  ship  hull,  defines  the  symmetry  plane  using  an  //-type  grid 
topology. 

For  grid  1  (i.e.,  finest  grid),  spacing  along  the  edges  of  the 
computational  block  was  controlled  by  specifying  the  grid  distri¬ 
bution  function  (e.g..  hyperbolic  tanh  or  geometric  series),  ^d 
spacing  at  the  endpoint(s),  and  number  of  points.  Grid  clustering 
was  used  near  the  bow  and  stem  in  the  ^-direction,  at  the  hull  in 
the  T^-direction,  and  near  the  free  surface  in  the  f-direction.  The 
faces  of  the  subblocks  were  smoothed  using  an  elliptic  solver  after 
which  the  coordinates  in  the  interior  were  obtained  using  transfi- 
nite  interpolation  from  the  block  faces.  The  three  subblocks  were 
then  joined  into  a  single  block  before  simulations  were  performed. 

Grid  2  was  generated  from  grid  1  by  using  the  same  grid  dis¬ 
tribution  function  and  by  increasing  the  fine  grid  spacing  at 
the  comers  of  the  blocks  and  decreasing  the  number  of  fine  grid 
computational  cells  in  each  coordinate  direction  by  a 

factor  ro 


Aaccj=rcAjCc, 

(17) 

N2=l+(N,-l)/rc 

(18) 

where  and  are  the  grid  spacing  and  and  N2  are  the 
number  of  points  on  grids  1  and  2,  respectively.  Finally,  the  faces 
of  the  blocks  for  grid  2  were  then  resmoothed  using  the  elliptic 
solver  and  coordinates  in  the  interior  were  obtained  using  transfi- 
nite  interpolation  (TFI). 

For  integer  grid  refinement  ratio,  this  procedure  results  in  a  set 
of  grids  with  systematic  refinement  (i.e.,  constant  refinement  ratio 
rc- Ajccj/Ajcc,^  Ajcc3/Axg^= However,  for  non¬ 
integer  refinement  ratio  Tq,  Eq.  (18)  yields  a  noninteger  grid 
number  N2  which  obviously  must  be  rounded  to  the  nearest  inte¬ 
ger.  This  results  in  a  difference  in  the  actual 
rn  refinement  ratio.  In  other  words, 

^TARGET 


’'O^CTUAL  integer[(Ni  - 1 

where  the  function  integer [  7”  is  used  to  denote  rounding  of  a 
real  number  to  the  nearest  integer  value. 

The  actual  and  target  refinement  ratios  were  compared  by  post¬ 
processing  grids  1-3.  Refinement  ratio  was  computed  at  grid 
points  along  the  intersection  of  the  free-  and  no-slip  surfaces 
which  shows  an  average  refinement  ratio  of  rG=1.39  between 
grids  1  and  2  and  re  =  1.44  between  grids  2  and  3.  These  values 
vary  by  2%  from  the  desired  target  value  of 
'^1.4142.  However,  inclusion  of  the  effects  of  nonuniform  grid 
refinement  ratio  for  such  small  differences  indicates  <\%  Sq  dif¬ 
ferences  in  the  estimates  for  numerical  uncertainties  (where  5^ 
is  the  solution  on  the  finest  grid),  warranting  the  assump¬ 
tion  of  uniform  grid  refinement  ratio.  Because  the  volume  grid  is 
obtained  using  TFI  from  the  elliptically  smoothed  faces  of  the 
blocks,  larger  deviations  from  uniform  grid  refinement  ratio  are 
possible  in  the  interior. 

Subsequent  work  has  developed  post-processing  tools  to  auto¬ 
mate  generation  of  multiple  grids  with  noninteger  grid  refinement 
ratio  and  to  ensure  uniform  grid  refinement  ratio  between  multiple 
grids  (Wilson  et  al.  [11]).  With  this  new  procedure,  the  finest  grid 
is  trivially  post-processed  using  a  sequence  of  three,  ID  interpo¬ 
lations  to  yield  a  medium  grid  with  noninteger  grid  refinement 
ratio.  This  is  in  contrast  to  the  approach  used  in  this  paper  as 
described  above  for  generation  of  the  medium  grid,  which  could 
become  prohibitively  difficult  and  time-consuming  for  large, 
multi-block  grid  systems  with  complex  geometry. 


6  Verification  and  Validation  of  Integral  Variable: 

Total  Resistance  Cx 

Friction  and  pressure  stresses  in  the  axial  direction  are  inte¬ 
grated  over  the  surface  area  of  the  Series  60  and  summed  to  yield 
the  total  resistance  coefficient  Cj  •  The  integration  is  performed  in 
post  processing  using  a  second-order  accurate  method  based  on 
the  trapezoidal  rule.  Results  are  analyzed  and  compared  for  both 
situations  in  which  simulation  numerical  uncertainty  is  taken  to  be 
stochastic  and  Usn  estimated  and  when  taken  to  be  deterministic 
and  and  Us^n  are  estimated  for  grid  studies  GSl  and  GS2,  as 
discussed  in  Section  5. 

Verification.  For  verification  of  the  uncorrected  solution,  U sn 
is  given  by  Eq.  (2),  while  for  the  corrected  solution,  and  Us^ 
are  given  by  Eqs.  (3)  and  (4),  respectively.  Since  steady-state 
simulations  are  performed,  only  contributions  due  to  iteration 
number  and  grid  size  are  considered  (e.g.,  Eq.  (2)  simplifies  to 
The  limiting  order  of  accuracy  is  estimated  using 
the  formal  order  of  accuracy  of  the  CFD  code  (i.e., 

=  2.0),  which  is  realized  for  solution  of  simplified  equations  with 
uniform  grids.  For  solution  of  the  RANS  equations  on  non- 
orthogonal  stretched  grids,  values  based  on  the  formal  order  may 
be  overly  optimistic  or  not  certain. 

Iterative  convergence  is  assessed  through  evaluation  of  the  Cj 
iteration  history  and  L2  norm  of  solution  changes  summed  over 
all  grid  points.  Figure  2  shows  a  portion  of  the  iterative  history  for 
grid  1 .  The  portion  shown  represents  a  computation  started  from  a 
previous  solution  and  does  not  include  the  total  iterative  history. 
Solution  change  drops  four  orders  of  magnitude  from  an  initial 
value  of  about  10”^  (not  shown)  to  a  final  value  of  10“*.  The 
variation  in  Cj  is  about  0.14%Si  (where  5]  is  the  solution  on  the 
finest  grid)  over  the  last  period  of  oscillation  (i.e.,  (//=0.07%Si). 
Iterative  uncertainty  is  estimated  as  half  the  range  of  the  maxi¬ 
mum  and  minimum  values  over  the  last  two  periods  of  oscillation 
(see  Fig.  2(c)).  Iterative  histories  for  grids  2-4  show  iterative 
uncertainties  of  about  0.02,  0.03,  and  0.01  %Si ,  respectively.  The 
level  of  iterative  uncertainties  Uj  for  grids  2-4  are  at  least  two 
orders  of  magnitude  less  than  the  corresponding  ^d  uncertainties 
Uc .  whereas  the  iterative  uncertainty  for  grid  1  is  only  one  order 
of  magnitude  smaller  than  the  grid  error.  For  all  four  grids,  the 
iteration  errors  and  uncertainties  are  assumed  to  be  negligible  in 
comparison  to  the  grid  errors  and  uncertainties  for  all  four  solu¬ 
tions  (i.e.,  Uc  such  that  Usn=  C/g)-  Since  iterative  errors  are 
negligible,  correction  of  solutions  for  iterative  error  is  not 
required. 

Grid  convergence  is  assessed  through  multiple  Cx  solutions  on 
four  systematically  refined  grids  with  constant  refinement  ratio 
(see  Section  5).  The  total  resistance  Cx  values  on  all  four  grids  are 
given  in  Table  2  along  with  computed  solution  changes  e  and 
benchmark  EFD  data  for  comparison.  The  convergence  ratio 
Rg(9)^  order  of  accuracy  and  correction  factor  Cg(14) 

are  shown  in  Table  3.  Since  0<Rk<^y  both  GSl  and  GS2  display 
monotonic  convergence  as  given  by  condition  (i%  Eq.  (10).  How¬ 
ever,  the  variability  is  large,  i.e.,  the  order  of  accuracy  for  GSl  is 
much  greater  than  for  GS2  and  both  are  greater  than  Pk^^^^Pk^f^ 
=  2.0.  As  a  result,  C^  is  greater  for  GSl  than  GS2.  Although  the 
fact  that  the  order  of  accuracy  does  not  approach  with  grid 
refinement  is  unexpected,  in  reality  the  estimation  of  pt^^^  for 
practical  applications  is  not  certain,  as  discussed  previously. 

Separate  verification  of  the  pressure  Cp  and  friction  Cp  com¬ 
ponents  of  C7<  =  C/»+  Cp)  helps  explain  the  variability  displayed 
by  Pc  for  Cx  between  GSl  and  GS2.  Note  that  in  model-ship 
testing  following  Fr  scaling,  Cx^f^.^-Cp-^Cp^^^  where  the  re¬ 
siduary  resistance  = 

given  by  the  ITTC  model-ship  correlation  line  evaluated  for 
model  and  full  scale  Re.  Table  2  includes  Cp  and  Cp  values  along 
with  Cp  and  Cp^^^  for  qualitative  comparison.  Cp  comprises 
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Table  3  Verification  of  Ct^xIO"^)  for  Series  60 


Iteration 


Fig.  2  Iteration  history  for  Series  60  on  grid  1:  (a)  solution 
change,  (b)  ship  forces-  Cf,Cp,  and  and  (c)  magnified  view 
of  total  resistance  Cj  over  last  two  periods  of  oscillation 


about  10%  Ct  and  is  convergent  towards  (for  GSl  within 

The  Cp  and  Cp  convergence  ratio  Rq,  order  of  ac¬ 
curacy  Pc  and  correction  factor  are  shown  in  Table  4.  For 
GSl,  Cp  is  grid  independent  iRc=  Uc^'O.O)  so  that  pq  cannot  be 
estimated  and  Cp  convergent  but  with  For  GS2,  both 

Cp  and  Cp  are  convergent  but  with  Pc^Pk^^  and  Pc^Pk^f^* 
spectively.  For  C/r ,  the  trends  with  regard  to  variability  are  simi- 

Table  2  Grid  convergence  study  for  total  Cr,  pressure  Cp, 
and  frictional  Cp  resistance  (x10“®)  for  Series  60 


S,  (grid  3) 

Sj  (grid  2) 

6.02 

■■ 

1.60 

C,  =  2.00 

0.0% 

3.69 

3.51 

3.45 

3.42 

H 

-11% 

-4.9% 

-1.7% 

mrc 

%s,. 


Study 

Rc 

Pc 

Co 

I 

(grids  1-3) 

0.21 

4.4 

3.7 

2 

(grids  2-4) 

0.44 

2.3 

1.3 

%S, 


lar  as  for  Cp  but  with  reduced  magnitude.  The  fact  that  the  Cp  is 
grid  independent  so  that  pc  cannot  be  estimated  using  RE  and  Cp 
is  monotonically  convergent  with  PG>Pk,f,  partially  explains  the 
variability  exhibited  in  Cj.  Such  complications  should  be  ex¬ 
pected  for  verification  of  integral  variables  comprised  of  multiple 
components,  especially  when  components  strongly  depend  on  dif¬ 
ferent  physics. 

Table  5  shows  the  estimated  grid  uncertainty  (/^(IS),  grid 
error  (13),  corrected  grid  uncertainty  ( 16),  and  numerical 
benchmark  Sc(l)*  Uncertainty  estimates  based  on  the  factor  of 
safety  approach  are  included  for  comparison.  The  grid  uncertainty 
is  less  for  GSl  than  GS2  and  the  values  (2%Sj  and  7  %Si, 
respectively)  are  reasonable  in  consideration  of  the  overall  num¬ 
ber  of  grid  points  used.  The  corrected  grid  uncertainty  is  also  less 
for  GSl  than  GS2,  but  the  difference  is  smaller  than  was  the  case 
for  Similarly  Sc,  which  according  to  RE 

should  be  grid  independent,  shows  3%  difference  when  comparing 
GSl  and  GS2.  The  factor  of  safety  approach  provides  less  conser¬ 
vative  estimates,  which  is  contrary  to  previous  experience  for  ana¬ 
lytical  benchmarks. 

Solutions  on  the  next  finer  grid  with  rQ=Vl  would  require 
2.4M  grid  points.  Uq  would  likely  be  reduced  to  the  current 
levels,  but  with  similar  or  greater  levels  for  Ui  making  it  difficult 
to  separate  iterative  and  grid  uncertainties.  Therefore,  from  a  re¬ 
source  point  of  view  it  may  be  sufficient  to  accept  the  current 
corrected  solution  Sc  on  the  finest  grid  with  associated  corrected 
grid  uncertainty  This  conclusion  is  supported  by  the  overall 
verification  results  (i.e.,  four  solutions  decrease  and  converge 
monotonically  with  grid  refinement  and  with  positive  ^J^O)  not¬ 
withstanding  the  variability  exhibited  in  pc  which  precludes  com¬ 
plete  confidence.  Nonetheless,  additional  solutions  are  desirable 
for  gaining  experience  and  an  understanding  of  the  nature  of  the 
asymptotic  range  for  practical  applications  and  hopefully  such  so¬ 
lutions  will  show  reduced  variability. 

Validation.  The  comparison  error  E=D-S{5),  validation 
uncertainty  C/v(6),  experimental  data  uncertainty  and  simu¬ 
lation  numerical  uncertainty  Usn(2)  are  shown  in  Table  6.  Un¬ 
certainty  due  to  the  use  of  previous  data  Uspd  was  not  considered. 


Table  4  Verification  of  Cp  and  Cp(x10~^)  for  Series  60 


Study 

c. 

_ _ 1 

Rc 

Pc 

Rc 

Pc 

c. 

J 

(grids  1-3) 

0.00 

‘ 

0.33 

3.2 

2.0 

2 

(grids  2-4) 

0.04 

9.5 

26 

0.40 

2.6 

1.5 

%s, 


Table  5  Errors  and  uncertainties  for  Cr<x10  ^)  for  Series  60 


Grid 

C,  unconected 

C,  corrected  | 

Vc(Cc} 

Vc  (Fsf 

Vc,(Fs> 

5c 

I 

2.1% 

0.5% 

1.2% 

0.9% 

0.1% 

4.99 

2 

6.7% 

5.6% 

5.5% 

1.1% 

1.1% 

4.83 

%s, 
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Table  6  Validation  of  uncorrected  total  resistance  for  Series 

60 


Grid 

E% 

Uy% 

1 

6.8 

3.1 

2.5 

1.9 

2 

5.7 

6.7 

2.5 

6.3 

%D, 


SO  the  validation  uncertainty  Note  that  since 

iterative  uncertainty  was  found  to  be  negligible  Usn- 
data  uncertainty  was  reported  to  be  t/p=2.5%D. 

For  grid  1,  \E\>Uv  such  that  Cj  is  not  validated  at  the  \E\ 
-1%D  level;  however,  from  an  uncertainty  standpoint,  the  sign 
and  magnitude  of  E  can  be  used  to  estimate  Ssma  and  make  mod¬ 
eling  improvements.  Further  reduction  of  t/y  requires  reduction  of 
VsN  and  Uo  since  they  are  of  similar  magnitude.  For  grid  2,  l£| 
<ljy  such  that  Ct  is  validated  but  at  the  larger  validation  uncer¬ 
tainty  level  of  Uy^(in%D  due  to  larger  Usn>  Such  large  Usn 
and  therefore  Uy  precludes  distinguishing  Ssma  ^oni  an  uncer¬ 
tainty  standpoint. 

Table  7  is  similar  to  Table  6,  but  for  the  corrected  comparison 
error  EcH)^  validation  uncertainty  Uy^{%)y  and  simulation  nu¬ 
merical  uncertainty  1/5^^ (4).  In  this  case,  \Ec\>Uy^  for  grids  1 
and  2  such  that  neither  is  validated  at  the  Uy— 2.6  and  2J%D 
levels,  respectively.  Us^n^^d  shows  that  reduction  of  l/p  by 
one  order  of  magnitude  is  required  to  reduce  U  y^  to  the  level  of 
Us^  and  leads  to  the  conclusion  that  the  comparison  error  is 

largely  due  to  modeling  errors  (i.e.,  Ec^  Ssma)’ 

The  overall  conclusion  for  V&V  of  resistance  is  that  Cf  is  not 
validated  on  the  finest  grid  due  to  modeling  errors  of  about  8%/). 
Likely  modeling  assumptions  to  consider  are  approximations  of 
static  sinkage  and  trim,  free  surface  boundary  conditions,  and  tur¬ 
bulence,  which  should  be  addressed  for  possible  validation  at 
about  the  3%D  level.  Reducing  levels  of  validation  uncertainty 
primarily  requires  reduction  in  experimental  uncertainties;  since, 
f/p(=2.5%D)>t/c^(  =  l%5i). 


7  Verification  and  Validation  of  a  Point  Variable: 
Wave  Profile  f 

The  wave  profile  is  defined  from  the  computed  wave  height  at 
the  intersection  of  the  free-  and  no-slip  hull  surfaces  from  0 
Wave  elevation  contours  for  the  entire  free-surface 
plane  are  shown  in  Figs.  l{b\  (d),  (f)  and  {h)  while  the  wave 
profile  on  all  four  grids  is  shown  in  Fig.  3  including  qualitative 
comparison  with  the  benchmark  EFD  data.  To  facilitate  the  com¬ 
parisons,  the  solutions  from  all  four  grids  are  interpolated  onto  the 
distribution  for  the  data.  Results  are  analyzed  and  compared  for 
both  situations  in  which  simulation  numerical  uncertainty  is  taken 
to  be  stochastic  and  Usn  estimated  and  when  taken  to  be  deter¬ 
ministic  and  and  Us^n  are  estimated  for  grid  studies  GSl  and 
GS2.  Both  point  distributions  and  profile-averages  of  errors  and 
uncertainties  are  discussed  for  verification  and  validation  of  the 
wave  profile. 

Verification.  Evaluation  of  convergence  ratio  Rq  (9),  order 
of  accuracy  Pg(12),  and  correction  factor  Cg(14),  for  point 
variables  can  be  problematic  when  solution  changes  821^  ^0^2 


Table  7  Validation  of  corrected  total  resistance  tor  Series  60 


Grid 

Ec% 

Uy,  % 

U^  % 

1 

7.9 

2.6 

2.5 

0.8 

2 

11 

2.7 

2.5 

1.0 

%D. 


-0.01  h 


Grid  1  (287x71x43) 
Grid  2  (201x51x31) 
Grid  3  (144x36x22) 
Grid  4  (101x26x16) 
Todaetal.  (1992) 


0.25  0.5  0.75  1 

x/L 


Fig.  3  Grid  study  for  Series  60  wave  profile 


both  go  to  zero  so  that  their  ratio  is  ill-defined.  To  overcome  this 
problem,  separate  L2  norms  of  Ccj,  and  egjj  are  used  to  define 
ratios  for  and  pc ,  i.e., 

</fc)  =  lkcJl2/ieGj2  (20) 

ln(||ec  Il2/l|scj,ll2) 

«'«> - 

where  <  >  and  ||  II2  are  used  to  denote  a  profile-averaged  quantity 
(with  ratio  of  solution  changes  based  on  L2  norms)  and  L2  norm, 
respectively. 

For  verification  of  the  uncorrected  solution,  Eq.  (15)  is  used  to 
estimate  distributions  of  Uc  each  point  from  the  local  solution 
change  6g2i»  Pg  estimated  from  Eq.  (21).  Similarly,  for 
the  corrected  solution,  {pc)  used  to  estimate  S'q  and  Uq^  at 
each  point  using  Eqs.  (13)  and  (16),  respectively.  An  L2  norm  of 
point  distributions  of  errors  and  uncertainties  are  then  used  to 
assess  verification  levels  and  to  judge  if  validation  has  been 
achieved  globally.  Iteration  errors  and  uncertainties  were  found  to 
be  negligible  in  comparison  to  the  grid  errors  and  uncertainties  for 
all  four  solutions,  i.e.,  U /<  Ug  such  that  U sn"^  ^ g  • 

The  profile-averaged  convergence  ratio  (Rg)»  order  of  accuracy 
(Pg)»  correction  factor  (Cg),  global  grid  uncertainty  Ug,  and 
corrected  grid  uncertainty  Ug^  are  shown  in  Table  8.  Both  GSl 
and  GS2  display  monotonic  convergence  condition  (i)  from  Eq. 
(10),  but  with  (pg)  for  GSl  greater  than  that  for  GS2  and  both 
less  than  which  in  this  case  is  consistent  with  expectations 
for  solutions  on  stretched,  curvilinear  grids.  The  levels  of  I/g 
Ug  for  GSl  are  1/2  those  for  GS2.  Variability  between  GSl  and 
GS2  is  absent  and  the  trends  are  consistent  with  expectation.  The 
distributions  for  Ug  and  Ug^  are  discussed  next  in  conjunction 

with  validation. 

Validation.  Validation  results  with  (Uy^Uy^)  and  {E^Eg) 
based  on  L2  norms  of  point  distributions  are  discussed  first,  fol¬ 
lowed  by  an  examination  of  the  actual  point  distributions.  Profile- 


Table  8  Profile-averaged  verification  results  for  wave  profile 
for  Series  60 


Study 

Rc 

Po 

Cc 

Uc 

^0, 

1 

(grids  I‘3) 

0.64 

1.3 

0.56 

2.0% 

0.9% 

2 

(grids  2-4} 

0.68 

l.I 

0.47 

4.1% 

2.2% 
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Table  9  Profile-averaged  validation  results  for  uncorrected 
wave  profile  for  Series  60 


Grid 

E% 

Uy% 

UsA 

} 

5.2 

4.2 

3.7 

2.0 

2 

5.6 

5.5 

3.7 

4.1 

averaged  validation  results  for  the  uncorrected  and  corrected  wave 
profile  are  given  in  Tables  9  and  10,  respectively.  Values  are  nor¬ 
malized  with  the  maximum  value  for  the  wave  profile 
=  0.014  and  the  uncertainty  in  the  data  is  3.1%  Por  grid  1, 
\E\>Uv  such  that  the  wave  profile  is  not  validated  at  the 
=  52%^max  level;  however  the  margin  to  achieve  validation  is 
small,  i.e.,  ^/d  is  roughly  twice  Usn-  Por  grid  2,  E 

--Uy  such  that  the  solution  is  nearly  validated  at  |El 
=  5.6%^max  level.  Uo  and  Usn  are  of  similar  magnitude. 

For  grids  1  and  2,  |E|  >  such  that  neither  is  validated  at  the 
I  Eel  =  5.6%  and  6.6%^;„ajr  levels,  respectively.  Here  again,  the 
margin  to  achieve  validation  is  fairly  small  (i.e.,  <2%  and 
<3%^^^x  for  grids  1  and  2,  respectively).  Up  is  roughly  twice 
and  four  times  for  grids  1  and  2,  respectively. 

Distributions  of  (E,  ±Uy)  and  (E^ ,  ±  t/y^)  x/L  are  shown 
in  Figs.  4(a,  b)  and  4(c,  d)  for  grids  1  and  2,  respectively.  When  E 
is  within  ±  Uy  or  Ec  is  within  -  Vy^,  the  solutions  are  validated 
at  the  levels  of  Uy  or  f/y^,  respectively.  Since  Uph  constant  the 
variations  in  Uy  and  Uy^  are  due  to  variations  in  Uc  and  Uq^.  In 
all  cases,  lack  of  validation  is  mainly  due  to  under  prediction  of 
wave  crests  and  troughs  in  the  simulations.  For  grid  1,  Uy^ 
^Up  since  Uq^<Up.  Although  modeling  errors  are  relatively 
small  (2%  and  3%^^^^  for  grids  1  and  2),  Fig.  4  clearly  indicates 
where  improvements  are  warranted. 

The  overall  conclusion  for  V&V  of  the  wave  profile  is  that  ^  is 
not  validated  due  to  modeling  errors  of  about  6%f^ojr »  however, 
in  this  case,  Uy  is  relatively  large  compared  to  E  making  the 


Table  10  Profile-averaged  validation  results  for  corrected 
wave  profile  for  Series  60 
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Ec% 
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3.7 

2.2 

0.2 

0.1 
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. 
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Fig.  4  Validation  of  wave  profile  for  Series  60:  (a)  and  (b)  grid 
1;  and  (c)  and  (d)  grid  2.  E  and  Ec  values  are  normalized  with 
the  maximum  value  for  the  wave  profile  ?m«x=0*0‘14 


margin  for  reduction  in  modeling  errors  small  (i.e.,  2%^fnax)' 
Modeling  improvements  should  be  made  to  increase  simulation 
predictions  at  crests  and  troughs  so  that  solutions  are  validated  at 
about  the  4%^^^^  level.  Reducing  the  level  of  validation  uncer¬ 
tainty  primarily  requires  reduction  in  experimental  uncertainties; 
since,  f/z>(  =  4%f„,,)> f/c^(  =  1 

8  Conclusions 

In  Part  2  of  this  two-part  paper,  V&V  results  were  presented  for 
RANS  simulation  of  the  Series  60  cargo/container  ship,  which 
provides  a  documented  solution  following  the  methodology  and 
procedures  presented  in  Part  1  (Stem  et  al.,  [2]).  Although  there 
are  many  issues  for  practical  applications,  the  methodology  and 
procedures  are  shown  to  be  successful  in  assessing  levels  of  veri¬ 
fication  and  validation  and  identifying  modeling  errors  in  some 
cases.  For  practical  applications,  solutions  are  far  from  the 
asymptotic  range;  therefore,  analysis  and  interpretation  of  the  re¬ 
sults  is  shown  to  be  important  in  assessing  variability  for  order  of 
accuracy,  levels  of  verification,  and  strategies  for  reducing  nu¬ 
merical  and  modeling  errors  and  uncertainties. 

Verification  of  the  resistance  (integral  variable)  and  wave  pro¬ 
file  (point  variable)  indicates  iterative  uncertainties  are  much  less 
than  grid  uncertainties,  which  are  about  2%5i  for  the  finest  grid 
with  0.9M  points.  Solutions  on  the  next  finer  grid  with  2.4M 
points  likely  will  reduce  simulation  numerical  uncertainties  to 
<  1  %S, ;  however,  iterative  errors  may  be  of  similar  order  mak¬ 
ing  it  difficult  to  separate  iterative  and  grid  uncertainties.  There¬ 
fore,  from  a  resource  point  of  view  it  may  be  sufficient  to  accept 
the  current  corrected  solution  (numerical  benchmark)  on  the  finest 
grid  with  corrected  grid  uncertainty  f/0^=l%S|.  Validation  of 
the  resistance  and  wave  profile  shows  modeling  errors  of  about 
8%D  and  6%^^axy  which  should  be  addressed  for  possible  vali¬ 
dation  at  the  3%D  and  4%^^^^  levels.  Reducing  the  level  of 
validation  uncertainty  primarily  requires  reduction  in  experimen¬ 
tal  uncertainties;  since,  Up>Uc^^  The  combination  of  a  reduc¬ 
tion  of  E  (through  reduction  of  modeling  errors)  and  Up  will 
produce  verified  and  validated  CFD  simulations  with  low  levels  of 
simulation  modeling  and  numerical  errors  and  uncertainties. 

Future  work  will  focus  on  both  fundamental  and  practical  V&V 
issues,  especially  for  applications  with  complex  geometry.  Funda¬ 
mental  issues  include  the  convergence  of  the  power  series  expan¬ 
sion  for  numerical  error,  assumptions  that  the  order  of  accuracy  is 
constant  for  ail  solutions,  and  estimation  of  the  limiting  order  of 
accuracy  in  the  definition  of  correction  factor.  Issues  with  practi¬ 
cal  applications  involve  observed  variability  in  order  of  accuracy 
estimates,  existence  of  the  asymptotic  range,  and  generation  of 
multiple  grids  and  solutions  with  non-integer  refinement  ratio.  The 
behavior  of  the  asymptotic  range  was  successfully  demonstrated 
in  Appendix  A  of  Part  1  for  simpler  analytical  benchmark  prob¬ 
lems.  However,  the  existence  and  behavior  of  the  asymptotic 
range  for  practical  problems  has  not  yet  been  demonstrated.  Such 
an  effort  would  require  solutions  on  many  more  grids  to  properly 
assess  variability  in  order  of  accuracy  estimates  and  finer  grids  to 
ensure  solutions  are  indeed  in  the  asymptotic  range.  More  experi¬ 
ence  is  needed  for  additional  practical  applications  and  different 
CFD  codes. 
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